[jira] [Comment Edited] (OAK-6829) ExternalPrivateStoreIT/ExternalSharedStoreIT.testSyncBigBlob failures
[
https://issues.apache.org/jira/browse/OAK-6829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222377#comment-16222377
]
Chetan Mehrotra edited comment on OAK-6829 at 10/27/17 1:41 PM:
Yes this was due to OAK-6873. Fixed this now with 1813536
was (Author: chetanm):
Yes this was due to OAK-6873. Fixed this now with
https://issues.apache.org/jira/browse/OAK-6829
> ExternalPrivateStoreIT/ExternalSharedStoreIT.testSyncBigBlob failures
> -
>
> Key: OAK-6829
> URL: https://issues.apache.org/jira/browse/OAK-6829
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Affects Versions: 1.7.9
>Reporter: Julian Reschke
>Assignee: Francesco Mari
> Fix For: 1.8, 1.7.11
>
> Attachments:
> TEST-org.apache.jackrabbit.oak.segment.standby.ExternalPrivateStoreIT.xml,
> TEST-org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT.xml,
> org.apache.jackrabbit.oak.segment.standby.ExternalPrivateStoreIT-output.txt,
> org.apache.jackrabbit.oak.segment.standby.ExternalPrivateStoreIT-output.txt,
> org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT-output.txt,
> org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT-output.txt
>
>
> {noformat}
> testSyncBigBlob(org.apache.jackrabbit.oak.segment.standby.ExternalPrivateStoreIT)
> Time elapsed: 27.921 sec <<< FAILURE!
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> Running org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT
> Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 93.353 sec
> <<< FAILURE! - in
> org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT
> testSyncBigBlob(org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT)
> Time elapsed: 30.772 sec <<< FAILURE!
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6829) ExternalPrivateStoreIT/ExternalSharedStoreIT.testSyncBigBlob failures
[
https://issues.apache.org/jira/browse/OAK-6829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222377#comment-16222377
]
Chetan Mehrotra commented on OAK-6829:
--
Yes this was due to OAK-6873. Fixed this now with
https://issues.apache.org/jira/browse/OAK-6829
> ExternalPrivateStoreIT/ExternalSharedStoreIT.testSyncBigBlob failures
> -
>
> Key: OAK-6829
> URL: https://issues.apache.org/jira/browse/OAK-6829
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Affects Versions: 1.7.9
>Reporter: Julian Reschke
>Assignee: Francesco Mari
> Fix For: 1.8, 1.7.11
>
> Attachments:
> TEST-org.apache.jackrabbit.oak.segment.standby.ExternalPrivateStoreIT.xml,
> TEST-org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT.xml,
> org.apache.jackrabbit.oak.segment.standby.ExternalPrivateStoreIT-output.txt,
> org.apache.jackrabbit.oak.segment.standby.ExternalPrivateStoreIT-output.txt,
> org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT-output.txt,
> org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT-output.txt
>
>
> {noformat}
> testSyncBigBlob(org.apache.jackrabbit.oak.segment.standby.ExternalPrivateStoreIT)
> Time elapsed: 27.921 sec <<< FAILURE!
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> Running org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT
> Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 93.353 sec
> <<< FAILURE! - in
> org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT
> testSyncBigBlob(org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT)
> Time elapsed: 30.772 sec <<< FAILURE!
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6873) UserInitializer should not use hard coded QueryIndexProvider
[
https://issues.apache.org/jira/browse/OAK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222375#comment-16222375
]
Chetan Mehrotra commented on OAK-6873:
--
Fixed that with 1813536
> UserInitializer should not use hard coded QueryIndexProvider
>
>
> Key: OAK-6873
> URL: https://issues.apache.org/jira/browse/OAK-6873
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: security
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8, 1.7.11
>
> Attachments: OAK-6873-v1.patch
>
>
> [UserInitializer|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/security/user/UserInitializer.java#L99]
> uses specific QueryIndexProvider
> {noformat}
> Root root = RootFactory.createSystemRoot(store, EmptyHook.INSTANCE,
> workspaceName,
> securityProvider,
> new CompositeQueryIndexProvider(new PropertyIndexProvider(), new
> NodeTypeIndexProvider()));
> {noformat}
> Due to this its not possible to use other implementation of
> QueryIndexProvider like Lucene. To allow using other index types it should
> instead use {{WhiteboardIndexProvider}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6873) UserInitializer should not use hard coded QueryIndexProvider
[
https://issues.apache.org/jira/browse/OAK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-6873.
--
Resolution: Fixed
Assignee: Chetan Mehrotra
Fix Version/s: 1.7.11
Thanks for the feedback. Applied the patch along with a testcase with 1813511
> UserInitializer should not use hard coded QueryIndexProvider
>
>
> Key: OAK-6873
> URL: https://issues.apache.org/jira/browse/OAK-6873
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: security
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8, 1.7.11
>
> Attachments: OAK-6873-v1.patch
>
>
> [UserInitializer|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/security/user/UserInitializer.java#L99]
> uses specific QueryIndexProvider
> {noformat}
> Root root = RootFactory.createSystemRoot(store, EmptyHook.INSTANCE,
> workspaceName,
> securityProvider,
> new CompositeQueryIndexProvider(new PropertyIndexProvider(), new
> NodeTypeIndexProvider()));
> {noformat}
> Due to this its not possible to use other implementation of
> QueryIndexProvider like Lucene. To allow using other index types it should
> instead use {{WhiteboardIndexProvider}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6873) UserInitializer should not use hard coded QueryIndexProvider
[
https://issues.apache.org/jira/browse/OAK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222034#comment-16222034
]
Chetan Mehrotra commented on OAK-6873:
--
bq. However, it's a bit weird that the getter (getIndexProvider) checks if the
setter is called first... I wouldn't expect that a getter throws an exception.
Sure, it's a private getter (right now), but still weird.
That is done to ensure that QueryIndexProviderAware contract is respected
otherwise it would lead to NPE later. With this check it is ensured that
initializer is called in proper way. We can move the check to caller also
> UserInitializer should not use hard coded QueryIndexProvider
>
>
> Key: OAK-6873
> URL: https://issues.apache.org/jira/browse/OAK-6873
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: security
>Reporter: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8
>
> Attachments: OAK-6873-v1.patch
>
>
> [UserInitializer|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/security/user/UserInitializer.java#L99]
> uses specific QueryIndexProvider
> {noformat}
> Root root = RootFactory.createSystemRoot(store, EmptyHook.INSTANCE,
> workspaceName,
> securityProvider,
> new CompositeQueryIndexProvider(new PropertyIndexProvider(), new
> NodeTypeIndexProvider()));
> {noformat}
> Due to this its not possible to use other implementation of
> QueryIndexProvider like Lucene. To allow using other index types it should
> instead use {{WhiteboardIndexProvider}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (OAK-6873) UserInitializer should not use hard coded QueryIndexProvider
[
https://issues.apache.org/jira/browse/OAK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220356#comment-16220356
]
Chetan Mehrotra edited comment on OAK-6873 at 10/27/17 8:38 AM:
[~stillalex] [~anchela] Work around supporting lucene based unique index is
done as part of OAK-6535. To move further and validate it fully this issue need
to be addressed.
-One approach can be to make {{UserInitializer}} {{WhiteboardAware}} and then
in {{Oak#createNewContentRepository}} pass the whiteboard instance and have
{{UserInitializer}} use it possible- Used a different appraoch to pass the
QueryIndexProvider explicitly by introducing a new {{QueryIndexProviderAware}}
interface. See attached patch
Thoughts?
was (Author: chetanm):
[~stillalex] [~anchela] Work around supporting lucene based unique index is
done as part of OAK-6535. To move further and validate it fully this issue need
to be addressed.
One approach can be to make {{UserInitializer}} {{WhiteboardAware}} and then in
{{Oak#createNewContentRepository}} pass the whiteboard instance and have
{{UserInitializer}} use it possible
Thoughts?
> UserInitializer should not use hard coded QueryIndexProvider
>
>
> Key: OAK-6873
> URL: https://issues.apache.org/jira/browse/OAK-6873
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: security
>Reporter: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8
>
> Attachments: OAK-6873-v1.patch
>
>
> [UserInitializer|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/security/user/UserInitializer.java#L99]
> uses specific QueryIndexProvider
> {noformat}
> Root root = RootFactory.createSystemRoot(store, EmptyHook.INSTANCE,
> workspaceName,
> securityProvider,
> new CompositeQueryIndexProvider(new PropertyIndexProvider(), new
> NodeTypeIndexProvider()));
> {noformat}
> Due to this its not possible to use other implementation of
> QueryIndexProvider like Lucene. To allow using other index types it should
> instead use {{WhiteboardIndexProvider}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6873) UserInitializer should not use hard coded QueryIndexProvider
[
https://issues.apache.org/jira/browse/OAK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-6873:
-
Attachment: OAK-6873-v1.patch
[possible patch|^OAK-6873-v1.patch] for the same. With this all test currently
pass so wanted to check if this approach looks fine
[~tmueller] [~stillalex] [~anchela] Please review
> UserInitializer should not use hard coded QueryIndexProvider
>
>
> Key: OAK-6873
> URL: https://issues.apache.org/jira/browse/OAK-6873
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: security
>Reporter: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8
>
> Attachments: OAK-6873-v1.patch
>
>
> [UserInitializer|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/security/user/UserInitializer.java#L99]
> uses specific QueryIndexProvider
> {noformat}
> Root root = RootFactory.createSystemRoot(store, EmptyHook.INSTANCE,
> workspaceName,
> securityProvider,
> new CompositeQueryIndexProvider(new PropertyIndexProvider(), new
> NodeTypeIndexProvider()));
> {noformat}
> Due to this its not possible to use other implementation of
> QueryIndexProvider like Lucene. To allow using other index types it should
> instead use {{WhiteboardIndexProvider}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6877) NodeBuilder#isReplaced behaves incorrectly for SegmentNodeStore
[
https://issues.apache.org/jira/browse/OAK-6877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-6877:
-
Attachment: OAK-6877-v1.patch
> NodeBuilder#isReplaced behaves incorrectly for SegmentNodeStore
> ---
>
> Key: OAK-6877
> URL: https://issues.apache.org/jira/browse/OAK-6877
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8
>
> Attachments: OAK-6877-v1.patch
>
>
> As seen in OAK-6876 NodeBuilder reports incorrectly for isReplaced call.
> Following test fails for Segment fixture
> {noformat}
> @Test
> public void isReplacedBehaviour() throws Exception{
> NodeBuilder nb = store.getRoot().builder();
> nb.child("a").setProperty("foo", "bar");
> store.merge(nb, EmptyHook.INSTANCE, CommitInfo.EMPTY);
> nb = store.getRoot().builder();
> nb.child("a").child("b");
> assertFalse(nb.getChildNode("a").isReplaced("foo"));
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (OAK-6876) IndexDisabler should not use NodeBuilder#isReplaced
[
https://issues.apache.org/jira/browse/OAK-6876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16221928#comment-16221928
]
Chetan Mehrotra edited comment on OAK-6876 at 10/27/17 8:30 AM:
Testcase below illustrates the difference in behaviour of
NodeBuilder#isReplaced with different stores
{noformat}
Index: oak-it/src/test/java/org/apache/jackrabbit/oak/api/NodeBuilderTest.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===
--- oak-it/src/test/java/org/apache/jackrabbit/oak/api/NodeBuilderTest.java
(revision )
+++ oak-it/src/test/java/org/apache/jackrabbit/oak/api/NodeBuilderTest.java
(revision )
@@ -0,0 +1,48 @@
+package org.apache.jackrabbit.oak.api;
+
+import org.apache.jackrabbit.oak.OakBaseTest;
+import org.apache.jackrabbit.oak.fixture.NodeStoreFixture;
+import org.apache.jackrabbit.oak.spi.commit.CommitInfo;
+import org.apache.jackrabbit.oak.spi.commit.EmptyHook;
+import org.apache.jackrabbit.oak.spi.state.NodeBuilder;
+import org.junit.Test;
+
+import static org.junit.Assert.assertFalse;
+
+public class NodeBuilderTest extends OakBaseTest {
+
+public NodeBuilderTest(NodeStoreFixture fixture) {
+super(fixture);
+}
+
+@Test
+public void isReplacedBehaviour() throws Exception{
+NodeBuilder nb = store.getRoot().builder();
+nb.child("a").setProperty("foo", "bar");
+
+store.merge(nb, EmptyHook.INSTANCE, CommitInfo.EMPTY);
+
+nb = store.getRoot().builder();
+nb.child("a").child("b");
+assertFalse(nb.getChildNode("a").isReplaced("foo"));
+}
+}
{noformat}
This test only fails for Segment. For others it passes
[~mduerig] [~mreutegg] Thoughts?
Update - Opened OAK-6877 to track this
was (Author: chetanm):
Testcase below illustrates the difference in behaviour of
NodeBuilder#isReplaced with different stores
{noformat}
Index: oak-it/src/test/java/org/apache/jackrabbit/oak/api/NodeBuilderTest.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===
--- oak-it/src/test/java/org/apache/jackrabbit/oak/api/NodeBuilderTest.java
(revision )
+++ oak-it/src/test/java/org/apache/jackrabbit/oak/api/NodeBuilderTest.java
(revision )
@@ -0,0 +1,48 @@
+package org.apache.jackrabbit.oak.api;
+
+import org.apache.jackrabbit.oak.OakBaseTest;
+import org.apache.jackrabbit.oak.fixture.NodeStoreFixture;
+import org.apache.jackrabbit.oak.spi.commit.CommitInfo;
+import org.apache.jackrabbit.oak.spi.commit.EmptyHook;
+import org.apache.jackrabbit.oak.spi.state.NodeBuilder;
+import org.junit.Test;
+
+import static org.junit.Assert.assertFalse;
+
+public class NodeBuilderTest extends OakBaseTest {
+
+public NodeBuilderTest(NodeStoreFixture fixture) {
+super(fixture);
+}
+
+@Test
+public void isReplacedBehaviour() throws Exception{
+NodeBuilder nb = store.getRoot().builder();
+nb.child("a").setProperty("foo", "bar");
+
+store.merge(nb, EmptyHook.INSTANCE, CommitInfo.EMPTY);
+
+nb = store.getRoot().builder();
+nb.child("a").child("b");
+assertFalse(nb.getChildNode("a").isReplaced("foo"));
+}
+}
{noformat}
This test only fails for Segment. For others it passes
[~mduerig] [~mreutegg] Thoughts?
> IndexDisabler should not use NodeBuilder#isReplaced
> ---
>
> Key: OAK-6876
> URL: https://issues.apache.org/jira/browse/OAK-6876
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8, 1.7.11
>
>
> {{IndexDisabler}} currently uses NodeBuilder#isReplaced method to check if
> "disableIndexesOnNextCycle" is set in current flow or not. This is used to
> ensure that disabling is not done in same cycle as the one where reindexing
> was done.
> {noformat}
> //Skip disabling for the cycle where reindexing just got completed
> if (idxBuilder.isReplaced(DISABLE_INDEXES_ON_NEXT_CYCLE)){
> return emptyList();
> }
> {noformat}
> This method though has issues as it would return true
> * If property is only modified. If property is added then it returns false
> * Even if the property is not added new it may return true if base state is
> different object. This happens to be case with SegmentNodeStore and not with
> others
> As a fix we should check explicitly with base state instead of using this api
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6877) NodeBuilder#isReplaced behaves incorrectly for SegmentNodeStore
Chetan Mehrotra created OAK-6877:
Summary: NodeBuilder#isReplaced behaves incorrectly for
SegmentNodeStore
Key: OAK-6877
URL: https://issues.apache.org/jira/browse/OAK-6877
Project: Jackrabbit Oak
Issue Type: Bug
Components: segment-tar
Reporter: Chetan Mehrotra
Priority: Minor
Fix For: 1.8
As seen in OAK-6876 NodeBuilder reports incorrectly for isReplaced call.
Following test fails for Segment fixture
{noformat}
@Test
public void isReplacedBehaviour() throws Exception{
NodeBuilder nb = store.getRoot().builder();
nb.child("a").setProperty("foo", "bar");
store.merge(nb, EmptyHook.INSTANCE, CommitInfo.EMPTY);
nb = store.getRoot().builder();
nb.child("a").child("b");
assertFalse(nb.getChildNode("a").isReplaced("foo"));
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6876) IndexDisabler should not use NodeBuilder#isReplaced
[
https://issues.apache.org/jira/browse/OAK-6876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-6876.
--
Resolution: Fixed
Fix Version/s: 1.7.11
> IndexDisabler should not use NodeBuilder#isReplaced
> ---
>
> Key: OAK-6876
> URL: https://issues.apache.org/jira/browse/OAK-6876
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8, 1.7.11
>
>
> {{IndexDisabler}} currently uses NodeBuilder#isReplaced method to check if
> "disableIndexesOnNextCycle" is set in current flow or not. This is used to
> ensure that disabling is not done in same cycle as the one where reindexing
> was done.
> {noformat}
> //Skip disabling for the cycle where reindexing just got completed
> if (idxBuilder.isReplaced(DISABLE_INDEXES_ON_NEXT_CYCLE)){
> return emptyList();
> }
> {noformat}
> This method though has issues as it would return true
> * If property is only modified. If property is added then it returns false
> * Even if the property is not added new it may return true if base state is
> different object. This happens to be case with SegmentNodeStore and not with
> others
> As a fix we should check explicitly with base state instead of using this api
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6876) IndexDisabler should not use NodeBuilder#isReplaced
[
https://issues.apache.org/jira/browse/OAK-6876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16221931#comment-16221931
]
Chetan Mehrotra commented on OAK-6876:
--
Change the logic with 1813481
> IndexDisabler should not use NodeBuilder#isReplaced
> ---
>
> Key: OAK-6876
> URL: https://issues.apache.org/jira/browse/OAK-6876
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8, 1.7.11
>
>
> {{IndexDisabler}} currently uses NodeBuilder#isReplaced method to check if
> "disableIndexesOnNextCycle" is set in current flow or not. This is used to
> ensure that disabling is not done in same cycle as the one where reindexing
> was done.
> {noformat}
> //Skip disabling for the cycle where reindexing just got completed
> if (idxBuilder.isReplaced(DISABLE_INDEXES_ON_NEXT_CYCLE)){
> return emptyList();
> }
> {noformat}
> This method though has issues as it would return true
> * If property is only modified. If property is added then it returns false
> * Even if the property is not added new it may return true if base state is
> different object. This happens to be case with SegmentNodeStore and not with
> others
> As a fix we should check explicitly with base state instead of using this api
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6876) IndexDisabler should not use NodeBuilder#isReplaced
[
https://issues.apache.org/jira/browse/OAK-6876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16221928#comment-16221928
]
Chetan Mehrotra commented on OAK-6876:
--
Testcase below illustrates the difference in behaviour of
NodeBuilder#isReplaced with different stores
{noformat}
Index: oak-it/src/test/java/org/apache/jackrabbit/oak/api/NodeBuilderTest.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===
--- oak-it/src/test/java/org/apache/jackrabbit/oak/api/NodeBuilderTest.java
(revision )
+++ oak-it/src/test/java/org/apache/jackrabbit/oak/api/NodeBuilderTest.java
(revision )
@@ -0,0 +1,48 @@
+package org.apache.jackrabbit.oak.api;
+
+import org.apache.jackrabbit.oak.OakBaseTest;
+import org.apache.jackrabbit.oak.fixture.NodeStoreFixture;
+import org.apache.jackrabbit.oak.spi.commit.CommitInfo;
+import org.apache.jackrabbit.oak.spi.commit.EmptyHook;
+import org.apache.jackrabbit.oak.spi.state.NodeBuilder;
+import org.junit.Test;
+
+import static org.junit.Assert.assertFalse;
+
+public class NodeBuilderTest extends OakBaseTest {
+
+public NodeBuilderTest(NodeStoreFixture fixture) {
+super(fixture);
+}
+
+@Test
+public void isReplacedBehaviour() throws Exception{
+NodeBuilder nb = store.getRoot().builder();
+nb.child("a").setProperty("foo", "bar");
+
+store.merge(nb, EmptyHook.INSTANCE, CommitInfo.EMPTY);
+
+nb = store.getRoot().builder();
+nb.child("a").child("b");
+assertFalse(nb.getChildNode("a").isReplaced("foo"));
+}
+}
{noformat}
This test only fails for Segment. For others it passes
[~mduerig] [~mreutegg] Thoughts?
> IndexDisabler should not use NodeBuilder#isReplaced
> ---
>
> Key: OAK-6876
> URL: https://issues.apache.org/jira/browse/OAK-6876
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8
>
>
> {{IndexDisabler}} currently uses NodeBuilder#isReplaced method to check if
> "disableIndexesOnNextCycle" is set in current flow or not. This is used to
> ensure that disabling is not done in same cycle as the one where reindexing
> was done.
> {noformat}
> //Skip disabling for the cycle where reindexing just got completed
> if (idxBuilder.isReplaced(DISABLE_INDEXES_ON_NEXT_CYCLE)){
> return emptyList();
> }
> {noformat}
> This method though has issues as it would return true
> * If property is only modified. If property is added then it returns false
> * Even if the property is not added new it may return true if base state is
> different object. This happens to be case with SegmentNodeStore and not with
> others
> As a fix we should check explicitly with base state instead of using this api
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6876) IndexDisabler should not use NodeBuilder#isReplaced
Chetan Mehrotra created OAK-6876:
Summary: IndexDisabler should not use NodeBuilder#isReplaced
Key: OAK-6876
URL: https://issues.apache.org/jira/browse/OAK-6876
Project: Jackrabbit Oak
Issue Type: Bug
Components: indexing
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Priority: Minor
Fix For: 1.8
{{IndexDisabler}} currently uses NodeBuilder#isReplaced method to check if
"disableIndexesOnNextCycle" is set in current flow or not. This is used to
ensure that disabling is not done in same cycle as the one where reindexing was
done.
{noformat}
//Skip disabling for the cycle where reindexing just got completed
if (idxBuilder.isReplaced(DISABLE_INDEXES_ON_NEXT_CYCLE)){
return emptyList();
}
{noformat}
This method though has issues as it would return true
* If property is only modified. If property is added then it returns false
* Even if the property is not added new it may return true if base state is
different object. This happens to be case with SegmentNodeStore and not with
others
As a fix we should check explicitly with base state instead of using this api
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6873) UserInitializer should not use hard coded QueryIndexProvider
[
https://issues.apache.org/jira/browse/OAK-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220356#comment-16220356
]
Chetan Mehrotra commented on OAK-6873:
--
[~stillalex] [~anchela] Work around supporting lucene based unique index is
done as part of OAK-6535. To move further and validate it fully this issue need
to be addressed.
One approach can be to make {{UserInitializer}} {{WhiteboardAware}} and then in
{{Oak#createNewContentRepository}} pass the whiteboard instance and have
{{UserInitializer}} use it possible
Thoughts?
> UserInitializer should not use hard coded QueryIndexProvider
>
>
> Key: OAK-6873
> URL: https://issues.apache.org/jira/browse/OAK-6873
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: security
>Reporter: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8
>
>
> [UserInitializer|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/security/user/UserInitializer.java#L99]
> uses specific QueryIndexProvider
> {noformat}
> Root root = RootFactory.createSystemRoot(store, EmptyHook.INSTANCE,
> workspaceName,
> securityProvider,
> new CompositeQueryIndexProvider(new PropertyIndexProvider(), new
> NodeTypeIndexProvider()));
> {noformat}
> Due to this its not possible to use other implementation of
> QueryIndexProvider like Lucene. To allow using other index types it should
> instead use {{WhiteboardIndexProvider}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6873) UserInitializer should not use hard coded QueryIndexProvider
Chetan Mehrotra created OAK-6873:
Summary: UserInitializer should not use hard coded
QueryIndexProvider
Key: OAK-6873
URL: https://issues.apache.org/jira/browse/OAK-6873
Project: Jackrabbit Oak
Issue Type: Improvement
Components: security
Reporter: Chetan Mehrotra
Priority: Minor
Fix For: 1.8
[UserInitializer|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/security/user/UserInitializer.java#L99]
uses specific QueryIndexProvider
{noformat}
Root root = RootFactory.createSystemRoot(store, EmptyHook.INSTANCE,
workspaceName,
securityProvider,
new CompositeQueryIndexProvider(new PropertyIndexProvider(), new
NodeTypeIndexProvider()));
{noformat}
Due to this its not possible to use other implementation of QueryIndexProvider
like Lucene. To allow using other index types it should instead use
{{WhiteboardIndexProvider}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6857) Lucene unique index should check path validity for uniqueness constraint
[
https://issues.apache.org/jira/browse/OAK-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-6857.
--
Resolution: Fixed
Fix Version/s: 1.7.11
> Lucene unique index should check path validity for uniqueness constraint
>
>
> Key: OAK-6857
> URL: https://issues.apache.org/jira/browse/OAK-6857
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8, 1.7.11
>
>
> The {{UniquenessConstraintValidator}} currently checks for any new index
> entry if any matching entry is found in primary (property) and secondary
> (lucene) index.
> This can lead to false positive where the entry in index is obsolete and due
> to async index lagging behind is yet not pruned. For e.g. if
> # Time T1 - path /a/@uuid=1 existed , present in lucene index
> # Time T2 - path /a removed (async index yet not catched up)
> # Time T3 - path /b/@uuid=1 being created.
> In this case save should pass. However as async index has yet not catched up
> it reports a constraint validation exception. As a fix
> {{UniquenessConstraintValidator}} should check if reported paths are valid
> wrt current revision
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6857) Lucene unique index should check path validity for uniqueness constraint
[
https://issues.apache.org/jira/browse/OAK-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220338#comment-16220338
]
Chetan Mehrotra commented on OAK-6857:
--
With 1813387 same check is also done for paths returned by the property index.
This is required as LuceneIndexEditor does not traverses the complete removed
subtree and instead relies on Lucene query to remove any indexed nodes from
that subtree. However that would cause issue with property index because it
would not detect the removal of unique properties.
So now validity check is also applied on paths returned from property index to
ensure that match with current repo state
> Lucene unique index should check path validity for uniqueness constraint
>
>
> Key: OAK-6857
> URL: https://issues.apache.org/jira/browse/OAK-6857
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8, 1.7.11
>
>
> The {{UniquenessConstraintValidator}} currently checks for any new index
> entry if any matching entry is found in primary (property) and secondary
> (lucene) index.
> This can lead to false positive where the entry in index is obsolete and due
> to async index lagging behind is yet not pruned. For e.g. if
> # Time T1 - path /a/@uuid=1 existed , present in lucene index
> # Time T2 - path /a removed (async index yet not catched up)
> # Time T3 - path /b/@uuid=1 being created.
> In this case save should pass. However as async index has yet not catched up
> it reports a constraint validation exception. As a fix
> {{UniquenessConstraintValidator}} should check if reported paths are valid
> wrt current revision
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6857) Lucene unique index should check path validity for uniqueness constraint
[
https://issues.apache.org/jira/browse/OAK-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220336#comment-16220336
]
Chetan Mehrotra commented on OAK-6857:
--
With 1813386 now UniquenessConstraintValidator would check if any path returned
by lucene index is valid wrt current head state or not
> Lucene unique index should check path validity for uniqueness constraint
>
>
> Key: OAK-6857
> URL: https://issues.apache.org/jira/browse/OAK-6857
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> The {{UniquenessConstraintValidator}} currently checks for any new index
> entry if any matching entry is found in primary (property) and secondary
> (lucene) index.
> This can lead to false positive where the entry in index is obsolete and due
> to async index lagging behind is yet not pruned. For e.g. if
> # Time T1 - path /a/@uuid=1 existed , present in lucene index
> # Time T2 - path /a removed (async index yet not catched up)
> # Time T3 - path /b/@uuid=1 being created.
> In this case save should pass. However as async index has yet not catched up
> it reports a constraint validation exception. As a fix
> {{UniquenessConstraintValidator}} should check if reported paths are valid
> wrt current revision
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6777) IndexReader closed exception in previous reader
[
https://issues.apache.org/jira/browse/OAK-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-6777:
-
Labels: (was: candidate_oak_1_6)
Fix Version/s: 1.6.6
Merged to 1.6 with 1813365,1813366
> IndexReader closed exception in previous reader
> ---
>
> Key: OAK-6777
> URL: https://issues.apache.org/jira/browse/OAK-6777
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.5
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8, 1.7.9, 1.6.6
>
> Attachments: OAK-6777-v1.patch
>
>
> Another variant of index reader closed exception as reported in OAK-6635 is
> seen for previous readers
> {noformat}
> 2017-10-03 21:21:41,810 ERROR NA [oak-lucene-16]
> o.a.j.o.p.i.l.h.DocumentQueue - Uncaught exception
> org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed
> at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:262)
> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:271)
> at
> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:250)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.NRTIndex.createReader(NRTIndex.java:257)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.NRTIndex.getPrimaryReader(NRTIndex.java:113)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.NRTIndex.getReaders(NRTIndex.java:148)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNodeManager.getNRTReaders(IndexNodeManager.java:243)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNodeManager.refreshReaders(IndexNodeManager.java:203)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNodeManager.access$100(IndexNodeManager.java:53)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNodeManager$1.run(IndexNodeManager.java:104)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.TimedRefreshPolicy.refreshIfRequired(TimedRefreshPolicy.java:59)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.TimedRefreshPolicy.refreshOnReadIfRequired(TimedRefreshPolicy.java:40)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNodeManager.acquire(IndexNodeManager.java:152)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.acquireIndexNode(IndexTracker.java:193)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.DocumentQueue.processDocs(DocumentQueue.java:235)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.DocumentQueue.addDocsToIndex(DocumentQueue.java:214)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.DocumentQueue.access$500(DocumentQueue.java:55)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.DocumentQueue$2$1.call(DocumentQueue.java:121)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.DocumentQueue$2$1.call(DocumentQueue.java:96)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6635) IndexReader closed exception in DocumentQueue
[
https://issues.apache.org/jira/browse/OAK-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-6635:
-
Labels: (was: candidate_oak_1_6)
Fix Version/s: 1.6.6
Merged to 1.6 with 1813364
> IndexReader closed exception in DocumentQueue
> -
>
> Key: OAK-6635
> URL: https://issues.apache.org/jira/browse/OAK-6635
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.5
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Blocker
> Fix For: 1.8, 1.7.8, 1.6.6
>
> Attachments: OAK-6635-v1.patch
>
>
> Even after fix for OAK-6572 following exception is being seen in some cases
> {noformat}
> 05.09.2017 23:06:15.489 *ERROR* [oak-lucene-3]
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.DocumentQueue Uncaught
> exception
> org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed
> at org.apache.lucene.index.IndexReader.decRef(IndexReader.java:233)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNodeManager.decrementSearcherUsageCount(IndexNodeManager.java:255)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNodeManager.access$400(IndexNodeManager.java:52)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNodeManager$IndexNodeImpl.release(IndexNodeManager.java:286)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.DocumentQueue.processDocs(DocumentQueue.java:275)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.DocumentQueue.addDocsToIndex(DocumentQueue.java:214)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.DocumentQueue.access$500(DocumentQueue.java:55)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.DocumentQueue$2$1.call(DocumentQueue.java:121)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.hybrid.DocumentQueue$2$1.call(DocumentQueue.java:96)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> This also causes the lock to be not released prevent async index from further
> updates
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6813) DocumentStore conditional remove: reduce set of supported conditions to what the Version GC needs
[ https://issues.apache.org/jira/browse/OAK-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218803#comment-16218803 ] Chetan Mehrotra commented on OAK-6813: -- +1. Being precise in what we require is important at this layer. So better to remove unused abstractions > DocumentStore conditional remove: reduce set of supported conditions to what > the Version GC needs > - > > Key: OAK-6813 > URL: https://issues.apache.org/jira/browse/OAK-6813 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: documentmk >Reporter: Julian Reschke >Assignee: Julian Reschke > Fix For: 1.8 > > Attachments: OAK-6813-simplified.diff, OAK-6813.diff > > > ...and verify in tests that implementations consistently reject other > variants. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6807) Query Recorder
[
https://issues.apache.org/jira/browse/OAK-6807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218791#comment-16218791
]
Chetan Mehrotra commented on OAK-6807:
--
Looks fine. Few comments
* Not sure how many concurrent query calls happen - If required we can use
ConcurrentHashMap with putIfAbsent and only handle the overflow case in
synchronized mode
* Log stats on shutdown to capture low usage mode
* Log line to have simple format for easier parsing
{noformat}
LOG.debug("|{}|{}", query, count);
{noformat}
* -Later a JMX view over the same would be useful- Leave that I would later add
a status printer for this so that it can get included in config status dump
(like support for index definitions)
[~catholicon] You have any wish list here!
> Query Recorder
> --
>
> Key: OAK-6807
> URL: https://issues.apache.org/jira/browse/OAK-6807
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
> Fix For: 1.8
>
>
> In order to manage indexes (e.g. find out which indexes are no longer needed,
> which properties don't need to be indexed any longer), we have an easy way to
> log all executed queries / query plans.
> Each entry only needs to be logged once (logging multiple times is OK, but
> ensure it's not logged to often). Different log levels can be used (e.g. log
> level "TRACE" logs more data, "DEBUG" less). For "DEBUG" level, overhead of
> logging should be minimal, so this can be kept enabled for a long time.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6857) Lucene unique index should check path validity for uniqueness constraint
Chetan Mehrotra created OAK-6857:
Summary: Lucene unique index should check path validity for
uniqueness constraint
Key: OAK-6857
URL: https://issues.apache.org/jira/browse/OAK-6857
Project: Jackrabbit Oak
Issue Type: Technical task
Components: lucene
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Fix For: 1.8
The {{UniquenessConstraintValidator}} currently checks for any new index entry
if any matching entry is found in primary (property) and secondary (lucene)
index.
This can lead to false positive where the entry in index is obsolete and due to
async index lagging behind is yet not pruned. For e.g. if
# Time T1 - path /a/@uuid=1 existed , present in lucene index
# Time T2 - path /a removed (async index yet not catched up)
# Time T3 - path /b/@uuid=1 being created.
In this case save should pass. However as async index has yet not catched up it
reports a constraint validation exception. As a fix
{{UniquenessConstraintValidator}} should check if reported paths are valid wrt
current revision
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6853) Oak run tooling to store reindexed property index in segment store
[
https://issues.apache.org/jira/browse/OAK-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-6853.
--
Resolution: Fixed
Fix Version/s: 1.7.10
Done with r1812868.
Now if {{--index-paths}} for reindex refers to a property index then that would
be stored in {{indexing-result/indexes/propertyIndexStore}} directory
> Oak run tooling to store reindexed property index in segment store
> --
>
> Key: OAK-6853
> URL: https://issues.apache.org/jira/browse/OAK-6853
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: run
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8, 1.7.10
>
>
> Currently oak-run index command only supports reindexing lucene indexes which
> are stored as lucene files on file system. Purpose of this task is to support
> storing property index nodestates in segment store.
> This would be useful to understand the actual storage size of property index
> on disk.
> This feature is purely one way and more for experimentation to understand the
> property index storage. Currently no support is provided for importing this
> generated property index back to target repository.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6853) Oak run tooling to store reindexed property index in segment store
Chetan Mehrotra created OAK-6853: Summary: Oak run tooling to store reindexed property index in segment store Key: OAK-6853 URL: https://issues.apache.org/jira/browse/OAK-6853 Project: Jackrabbit Oak Issue Type: New Feature Components: run Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Fix For: 1.8 Currently oak-run index command only supports reindexing lucene indexes which are stored as lucene files on file system. Purpose of this task is to support storing property index nodestates in segment store. This would be useful to understand the actual storage size of property index on disk. This feature is purely one way and more for experimentation to understand the property index storage. Currently no support is provided for importing this generated property index back to target repository. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (OAK-6851) Disabled index with reindex flag enabled should not lead to reindex report
[
https://issues.apache.org/jira/browse/OAK-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-6851.
--
Resolution: Fixed
Fix Version/s: 1.7.10
Fixed with 1812734
> Disabled index with reindex flag enabled should not lead to reindex report
> --
>
> Key: OAK-6851
> URL: https://issues.apache.org/jira/browse/OAK-6851
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8, 1.7.10
>
>
> Currently if any index is disabled but has reindex flag enabled then it leads
> to following log entry for each commit (assuming disabled index was sync)
> {noformat}
> 20.10.2017 13:15:40.031 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
> 20.10.2017 13:15:40.046 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
> 20.10.2017 13:15:40.046 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
> 20.10.2017 13:15:40.056 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
> 20.10.2017 13:15:40.056 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
> 20.10.2017 13:15:40.068 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
> 20.10.2017 13:15:40.068 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
> 20.10.2017 13:15:40.076 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
> 20.10.2017 13:15:40.076 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
> 20.10.2017 13:15:40.092 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
> 20.10.2017 13:15:40.092 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
> 20.10.2017 13:15:40.103 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
> 20.10.2017 13:15:40.103 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
> 20.10.2017 13:15:40.123 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
> 20.10.2017 13:15:40.123 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
> 20.10.2017 13:15:40.131 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
> 20.10.2017 13:15:40.131 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
> 20.10.2017 13:15:40.137 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
> 20.10.2017 13:15:40.137 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
> 20.10.2017 13:15:40.143 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
> 20.10.2017 13:15:40.143 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
> 20.10.2017 13:15:40.149 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
> 20.10.2017 13:15:40.149 *INFO* [FelixStartLevel]
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing complet
> {noformat}
> The empty lines are due to report not logging details if reindex index
> entryCount is 0. As a fix reindex logic should totally ignore disabled indexes
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6851) Disabled index with reindex flag enabled should not lead to reindex report
Chetan Mehrotra created OAK-6851:
Summary: Disabled index with reindex flag enabled should not lead
to reindex report
Key: OAK-6851
URL: https://issues.apache.org/jira/browse/OAK-6851
Project: Jackrabbit Oak
Issue Type: Improvement
Components: indexing
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Priority: Minor
Fix For: 1.8
Currently if any index is disabled but has reindex flag enabled then it leads
to following log entry for each commit (assuming disabled index was sync)
{noformat}
20.10.2017 13:15:40.031 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
20.10.2017 13:15:40.046 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
20.10.2017 13:15:40.046 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
20.10.2017 13:15:40.056 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
20.10.2017 13:15:40.056 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
20.10.2017 13:15:40.068 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
20.10.2017 13:15:40.068 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
20.10.2017 13:15:40.076 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
20.10.2017 13:15:40.076 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
20.10.2017 13:15:40.092 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
20.10.2017 13:15:40.092 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
20.10.2017 13:15:40.103 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
20.10.2017 13:15:40.103 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
20.10.2017 13:15:40.123 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
20.10.2017 13:15:40.123 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
20.10.2017 13:15:40.131 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
20.10.2017 13:15:40.131 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
20.10.2017 13:15:40.137 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
20.10.2017 13:15:40.137 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
20.10.2017 13:15:40.143 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
20.10.2017 13:15:40.143 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing completed
20.10.2017 13:15:40.149 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Indexing report
20.10.2017 13:15:40.149 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Reindexing complet
{noformat}
The empty lines are due to report not logging details if reindex index
entryCount is 0. As a fix reindex logic should totally ignore disabled indexes
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6849) Avoid creating empty properties node under indexRules
[ https://issues.apache.org/jira/browse/OAK-6849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-6849. -- Resolution: Fixed Fixing it now with 1812733. All tests pass now > Avoid creating empty properties node under indexRules > - > > Key: OAK-6849 > URL: https://issues.apache.org/jira/browse/OAK-6849 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.8, 1.7.10 > > > IndexDefinitionBuilder currently creates an empty properties node under > indexRule. With OAK-6832 we would be creating empty indexRules for nodetype > index we should avoid this empty properties node -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (OAK-6849) Avoid creating empty properties node under indexRules
[ https://issues.apache.org/jira/browse/OAK-6849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-6849. -- Resolution: Fixed Fix Version/s: 1.7.10 Done with 1812709 > Avoid creating empty properties node under indexRules > - > > Key: OAK-6849 > URL: https://issues.apache.org/jira/browse/OAK-6849 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.8, 1.7.10 > > > IndexDefinitionBuilder currently creates an empty properties node under > indexRule. With OAK-6832 we would be creating empty indexRules for nodetype > index we should avoid this empty properties node -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-6849) Avoid creating empty properties node under indexRules
Chetan Mehrotra created OAK-6849: Summary: Avoid creating empty properties node under indexRules Key: OAK-6849 URL: https://issues.apache.org/jira/browse/OAK-6849 Project: Jackrabbit Oak Issue Type: Improvement Components: lucene Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Priority: Minor Fix For: 1.8 IndexDefinitionBuilder currently creates an empty properties node under indexRule. With OAK-6832 we would be creating empty indexRules for nodetype index we should avoid this empty properties node -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-3878) Avoid caching of NodeDocument while iterating in BlobReferenceIterator
[
https://issues.apache.org/jira/browse/OAK-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207252#comment-16207252
]
Chetan Mehrotra commented on OAK-3878:
--
[~reschke] Purpose of this api is for batch processing and those caller would
not be using the regular api. So its special purpose api not for generic use
> Avoid caching of NodeDocument while iterating in BlobReferenceIterator
> --
>
> Key: OAK-3878
> URL: https://issues.apache.org/jira/browse/OAK-3878
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8
>
>
> {{BlobReferenceIterator}} in DocumentMK makes use of {{DocumentStore}} API to
> query the NodeDocument. This would cause all those NodeDocuments to be added
> to cache in DocumentStore. Due to this when blob gc is running cache usage
> would not be that effective due to all the associated churn.
> As these NodeDocument are only required for BlobGC logic and its not expected
> that this document would read again soon it would be better to skip caching
> of these documents within DocumentStore
> Similar requirement exist in VersionGC logic but there we use direct store
> based API which does not add such documents to the cache
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6832) Synchronous nodetype lucene index support
[ https://issues.apache.org/jira/browse/OAK-6832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-6832. -- Resolution: Fixed Fix Version/s: 1.7.10 > Synchronous nodetype lucene index support > - > > Key: OAK-6832 > URL: https://issues.apache.org/jira/browse/OAK-6832 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8, 1.7.10 > > > Implement support for synchronous nodetype indexes. This would be built on > top of support added in OAK-6831 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (OAK-6831) Nodetype index support in Lucene Index
[
https://issues.apache.org/jira/browse/OAK-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-6831.
--
Resolution: Fixed
Fix Version/s: 1.7.10
> Nodetype index support in Lucene Index
> --
>
> Key: OAK-6831
> URL: https://issues.apache.org/jira/browse/OAK-6831
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8, 1.7.10
>
>
> Lucene indexes currently support nodetype index in some form by specifying a
> property definition for "jcr:primaryType" with propertyIndex=true. However
> this can cause issue if such rules are mixed with other rules.
> For supporting usecase where same lucene index supports multiple nodetype
> rules and can be used as pure nodetype index we should have a explicit
> support for indexing nodetypes.
> *Proposal*
> Any indexRule would support following properties
> * {{nodeTypeIndex}} - Boolean indicating if this rule is for nodetype indexing
> * {{sync}} - If nodetype index is sync
> if {{nodeTypeIndex}} is enabled then any explicit property definition would
> be ignored. With this mode following index definition would be safe to use
> (updated definition per final implementation)
> {noformat}
> /oak:index/nodeTypeLucene
> - jcr:primaryType = "oak:QueryIndexDefinition"
> - compatVersion = 2
> - type = "lucene"
> - async = "async"
> - nodeTypeIndex = true
> + indexRules
> + nt:file
> - sync = true
> + app:Component
> {noformat}
> Here the rule order would not cause any affect as for any matching rule the
> nodes primary and mixin types would be indexed
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6832) Synchronous nodetype lucene index support
[ https://issues.apache.org/jira/browse/OAK-6832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205722#comment-16205722 ] Chetan Mehrotra commented on OAK-6832: -- Done the implementation with 1812277 > Synchronous nodetype lucene index support > - > > Key: OAK-6832 > URL: https://issues.apache.org/jira/browse/OAK-6832 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8, 1.7.10 > > > Implement support for synchronous nodetype indexes. This would be built on > top of support added in OAK-6831 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-6835) nodetype lucene index can create unusable nodes under index structures
Chetan Mehrotra created OAK-6835:
Summary: nodetype lucene index can create unusable nodes under
index structures
Key: OAK-6835
URL: https://issues.apache.org/jira/browse/OAK-6835
Project: Jackrabbit Oak
Issue Type: Bug
Components: property-index
Reporter: Chetan Mehrotra
Assignee: Vikas Saurabh
Priority: Minor
Fix For: 1.8
{{nodetype}} index is just a special property index definition with definition
declaring to index {{jcr:primaryType}} and {{jcr:mixinTypes}}.
Since, it's just property index, we can specify {{declaringNodeTypes}} too to
filter which type of nodes get indexed.
On query side, declaringNodeTypes are used to check whether the index can be
used at all or not.
Now, for nodetype, if node being indexed passes declaringNodeType filter
(either primary type or mixin matches), then all its mixins and primaryType
gets indexed irrespective of whether declaringNodeType contains it or not.
This is perfectly correct behavior from property index point of view. But, in
this regards, we should treat nodetype index to be special and index only those
property values that are part of declaring node types.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6835) nodetype lucene index can create unusable nodes under index structures
[
https://issues.apache.org/jira/browse/OAK-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-6835:
-
Component/s: (was: property-index)
lucene
> nodetype lucene index can create unusable nodes under index structures
> --
>
> Key: OAK-6835
> URL: https://issues.apache.org/jira/browse/OAK-6835
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Vikas Saurabh
>Priority: Minor
> Fix For: 1.8
>
>
> {{nodetype}} index is just a special property index definition with
> definition declaring to index {{jcr:primaryType}} and {{jcr:mixinTypes}}.
> Since, it's just property index, we can specify {{declaringNodeTypes}} too to
> filter which type of nodes get indexed.
> On query side, declaringNodeTypes are used to check whether the index can be
> used at all or not.
> Now, for nodetype, if node being indexed passes declaringNodeType filter
> (either primary type or mixin matches), then all its mixins and primaryType
> gets indexed irrespective of whether declaringNodeType contains it or not.
> This is perfectly correct behavior from property index point of view. But, in
> this regards, we should treat nodetype index to be special and index only
> those property values that are part of declaring node types.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6831) Nodetype index support in Lucene Index
[
https://issues.apache.org/jira/browse/OAK-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205716#comment-16205716
]
Chetan Mehrotra commented on OAK-6831:
--
With these changes the nodetype support is now consistent. Whats pending is
OAK-6835
> Nodetype index support in Lucene Index
> --
>
> Key: OAK-6831
> URL: https://issues.apache.org/jira/browse/OAK-6831
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> Lucene indexes currently support nodetype index in some form by specifying a
> property definition for "jcr:primaryType" with propertyIndex=true. However
> this can cause issue if such rules are mixed with other rules.
> For supporting usecase where same lucene index supports multiple nodetype
> rules and can be used as pure nodetype index we should have a explicit
> support for indexing nodetypes.
> *Proposal*
> Any indexRule would support following properties
> * {{nodeTypeIndex}} - Boolean indicating if this rule is for nodetype indexing
> * {{sync}} - If nodetype index is sync
> if {{nodeTypeIndex}} is enabled then any explicit property definition would
> be ignored. With this mode following index definition would be safe to use
> (updated definition per final implementation)
> {noformat}
> /oak:index/nodeTypeLucene
> - jcr:primaryType = "oak:QueryIndexDefinition"
> - compatVersion = 2
> - type = "lucene"
> - async = "async"
> - nodeTypeIndex = true
> + indexRules
> + nt:file
> - sync = true
> + app:Component
> {noformat}
> Here the rule order would not cause any affect as for any matching rule the
> nodes primary and mixin types would be indexed
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (OAK-6835) nodetype lucene index can create unusable nodes under index structures
[
https://issues.apache.org/jira/browse/OAK-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra reassigned OAK-6835:
Assignee: Chetan Mehrotra (was: Vikas Saurabh)
> nodetype lucene index can create unusable nodes under index structures
> --
>
> Key: OAK-6835
> URL: https://issues.apache.org/jira/browse/OAK-6835
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8
>
>
> {{nodetype}} index is just a special property index definition with
> definition declaring to index {{jcr:primaryType}} and {{jcr:mixinTypes}}.
> Since, it's just property index, we can specify {{declaringNodeTypes}} too to
> filter which type of nodes get indexed.
> On query side, declaringNodeTypes are used to check whether the index can be
> used at all or not.
> Now, for nodetype, if node being indexed passes declaringNodeType filter
> (either primary type or mixin matches), then all its mixins and primaryType
> gets indexed irrespective of whether declaringNodeType contains it or not.
> This is perfectly correct behavior from property index point of view. But, in
> this regards, we should treat nodetype index to be special and index only
> those property values that are part of declaring node types.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6831) Nodetype index support in Lucene Index
[
https://issues.apache.org/jira/browse/OAK-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205687#comment-16205687
]
Chetan Mehrotra commented on OAK-6831:
--
with r1812276 now if any indexDefinition has a property definition only for
"jcr:primaryType" then it would again synthesize a definition for
'jcr:mixinTypes'. This would ensure that nodetype query for single index rule
entries work as expected
> Nodetype index support in Lucene Index
> --
>
> Key: OAK-6831
> URL: https://issues.apache.org/jira/browse/OAK-6831
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> Lucene indexes currently support nodetype index in some form by specifying a
> property definition for "jcr:primaryType" with propertyIndex=true. However
> this can cause issue if such rules are mixed with other rules.
> For supporting usecase where same lucene index supports multiple nodetype
> rules and can be used as pure nodetype index we should have a explicit
> support for indexing nodetypes.
> *Proposal*
> Any indexRule would support following properties
> * {{nodeTypeIndex}} - Boolean indicating if this rule is for nodetype indexing
> * {{sync}} - If nodetype index is sync
> if {{nodeTypeIndex}} is enabled then any explicit property definition would
> be ignored. With this mode following index definition would be safe to use
> (updated definition per final implementation)
> {noformat}
> /oak:index/nodeTypeLucene
> - jcr:primaryType = "oak:QueryIndexDefinition"
> - compatVersion = 2
> - type = "lucene"
> - async = "async"
> - nodeTypeIndex = true
> + indexRules
> + nt:file
> - sync = true
> + app:Component
> {noformat}
> Here the rule order would not cause any affect as for any matching rule the
> nodes primary and mixin types would be indexed
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (OAK-6831) Nodetype index support in Lucene Index
[
https://issues.apache.org/jira/browse/OAK-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205678#comment-16205678
]
Chetan Mehrotra edited comment on OAK-6831 at 10/16/17 10:31 AM:
-
With r1812274 now if {{nodeTypeIndex}} is true the IndexDefinition would
synthesize property definitions on 'jcr:primaryType' and 'jcr:mixinTypes'.
Indexing and query code would only rely on property definition entries and not
on "nodeTypeIndex" property.
Further with {{nodeTypeIndex}} set IndexDefinition would ignore any property
definition and aggregate definition
was (Author: chetanm):
With r1812274 now if {{nodeTypeIndex}} is true the IndexDefinition would
synthesize property definitions on 'jcr:primaryType' and 'jcr:mixinTypes'.
Indexing and query code would only rely on property definition entries and not
on "nodeTypeIndex" property
> Nodetype index support in Lucene Index
> --
>
> Key: OAK-6831
> URL: https://issues.apache.org/jira/browse/OAK-6831
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> Lucene indexes currently support nodetype index in some form by specifying a
> property definition for "jcr:primaryType" with propertyIndex=true. However
> this can cause issue if such rules are mixed with other rules.
> For supporting usecase where same lucene index supports multiple nodetype
> rules and can be used as pure nodetype index we should have a explicit
> support for indexing nodetypes.
> *Proposal*
> Any indexRule would support following properties
> * {{nodeTypeIndex}} - Boolean indicating if this rule is for nodetype indexing
> * {{sync}} - If nodetype index is sync
> if {{nodeTypeIndex}} is enabled then any explicit property definition would
> be ignored. With this mode following index definition would be safe to use
> (updated definition per final implementation)
> {noformat}
> /oak:index/nodeTypeLucene
> - jcr:primaryType = "oak:QueryIndexDefinition"
> - compatVersion = 2
> - type = "lucene"
> - async = "async"
> - nodeTypeIndex = true
> + indexRules
> + nt:file
> - sync = true
> + app:Component
> {noformat}
> Here the rule order would not cause any affect as for any matching rule the
> nodes primary and mixin types would be indexed
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6831) Nodetype index support in Lucene Index
[
https://issues.apache.org/jira/browse/OAK-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-6831:
-
Description:
Lucene indexes currently support nodetype index in some form by specifying a
property definition for "jcr:primaryType" with propertyIndex=true. However this
can cause issue if such rules are mixed with other rules.
For supporting usecase where same lucene index supports multiple nodetype rules
and can be used as pure nodetype index we should have a explicit support for
indexing nodetypes.
*Proposal*
Any indexRule would support following properties
* {{nodeTypeIndex}} - Boolean indicating if this rule is for nodetype indexing
* {{sync}} - If nodetype index is sync
if {{nodeTypeIndex}} is enabled then any explicit property definition would be
ignored. With this mode following index definition would be safe to use
(updated definition per final implementation)
{noformat}
/oak:index/nodeTypeLucene
- jcr:primaryType = "oak:QueryIndexDefinition"
- compatVersion = 2
- type = "lucene"
- async = "async"
- nodeTypeIndex = true
+ indexRules
+ nt:file
- sync = true
+ app:Component
{noformat}
Here the rule order would not cause any affect as for any matching rule the
nodes primary and mixin types would be indexed
was:
Lucene indexes currently support nodetype index in some form by specifying a
property definition for "jcr:primaryType" with propertyIndex=true. However this
can cause issue if such rules are mixed with other rules.
For supporting usecase where same lucene index supports multiple nodetype rules
and can be used as pure nodetype index we should have a explicit support for
indexing nodetypes.
*Proposal*
Any indexRule would support following properties
* {{nodeTypeIndex}} - Boolean indicating if this rule is for nodetype indexing
* {{sync}} - If nodetype index is sync
if {{nodeTypeIndex}} is enabled then any explicit property definition would be
ignored. With this mode following index definition would be safe to use
{noformat}
/oak:index/nodeTypeLucene
- jcr:primaryType = "oak:QueryIndexDefinition"
- compatVersion = 2
- type = "lucene"
- async = "async"
+ indexRules
+ nt:file
- nodeTypeIndex = true
+ app:Component
- nodeTypeIndex = true
{noformat}
Here the rule order would not cause any affect as for any matching rule the
nodes primary and mixin types would be indexed
> Nodetype index support in Lucene Index
> --
>
> Key: OAK-6831
> URL: https://issues.apache.org/jira/browse/OAK-6831
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> Lucene indexes currently support nodetype index in some form by specifying a
> property definition for "jcr:primaryType" with propertyIndex=true. However
> this can cause issue if such rules are mixed with other rules.
> For supporting usecase where same lucene index supports multiple nodetype
> rules and can be used as pure nodetype index we should have a explicit
> support for indexing nodetypes.
> *Proposal*
> Any indexRule would support following properties
> * {{nodeTypeIndex}} - Boolean indicating if this rule is for nodetype indexing
> * {{sync}} - If nodetype index is sync
> if {{nodeTypeIndex}} is enabled then any explicit property definition would
> be ignored. With this mode following index definition would be safe to use
> (updated definition per final implementation)
> {noformat}
> /oak:index/nodeTypeLucene
> - jcr:primaryType = "oak:QueryIndexDefinition"
> - compatVersion = 2
> - type = "lucene"
> - async = "async"
> - nodeTypeIndex = true
> + indexRules
> + nt:file
> - sync = true
> + app:Component
> {noformat}
> Here the rule order would not cause any affect as for any matching rule the
> nodes primary and mixin types would be indexed
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-6820:
-
Labels: docs-impacting (was: )
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Labels: docs-impacting
> Fix For: 1.8, 1.7.10
>
> Attachments: OAK-6820-v1.patch, OAK-6820-v2.patch
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-6820.
--
Resolution: Fixed
Fix Version/s: 1.7.10
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Labels: docs-impacting
> Fix For: 1.8, 1.7.10
>
> Attachments: OAK-6820-v1.patch, OAK-6820-v2.patch
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205417#comment-16205417
]
Chetan Mehrotra commented on OAK-6820:
--
Applied the patch with 1812242
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
> Attachments: OAK-6820-v1.patch, OAK-6820-v2.patch
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-6820:
-
Attachment: OAK-6820-v2.patch
[updated patch|^OAK-6820-v2.patch] based on above approach
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
> Attachments: OAK-6820-v1.patch, OAK-6820-v2.patch
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (OAK-6831) Nodetype index support in Lucene Index
[
https://issues.apache.org/jira/browse/OAK-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205000#comment-16205000
]
Chetan Mehrotra edited comment on OAK-6831 at 10/15/17 3:03 AM:
bq. I'm not very comfortable with the node type definition. I'd much rather
prefer that we bring in a new index type - internally we can, of course, map it
to the IndexDefinition you described above.
That would be a bigger change and something which I feel we do not require now.
What about we move this {{nodeTypeIndex}} to top level and not at individual
rule level. That would allow us to ensure that no other rule defines property
definitions. So no way to get into ambiguous state
bq. is that it would most likely index data that is not expected in the index
(e.g. OAK-4653 - what nodetype index does). Basically, we should work with all
rules instead of what getApplicableIndexingRule reports.
Note that with current design for following nodetypes
{noformat}
[oak:TestMixA]
mixin
[oak:TestSuperType]
- * (UNDEFINED) multiple
[oak:TestTypeA] > oak:TestSuperType
- * (UNDEFINED) multiple
[oak:TestTypeB] > oak:TestSuperType, oak:TestMixA
- * (UNDEFINED) multiple
{noformat}
If you enable {{nodeTypeIndex}} to true for say {{oak:TestMixA}} then that
would create 2 internal indexRules for {{oak:TestMixA}} and {{oak:TestTypeB}}
(where second is clone of first). -So to handle OAK-4653 what we can do is just
index the indexRule name and not read primaryType and mixinTypes property
values-
Now I get what you mean by intersection - A node being indexed may have been
indexed due to rule r1 but same index definition may be having other nodeType
rules which this node qualifies. So yes to be more precise we would need to
index the intersection
was (Author: chetanm):
bq. I'm not very comfortable with the node type definition. I'd much rather
prefer that we bring in a new index type - internally we can, of course, map it
to the IndexDefinition you described above.
That would be a bigger change and something which I feel we do not require now.
What about we move this {{nodeTypeIndex}} to top level and not at individual
rule level. That would allow us to ensure that no other rule defines property
definitions. So no way to get into ambiguous state
bq. is that it would most likely index data that is not expected in the index
(e.g. OAK-4653 - what nodetype index does). Basically, we should work with all
rules instead of what getApplicableIndexingRule reports.
Note that with current design for following nodetypes
{noformat}
[oak:TestMixA]
mixin
[oak:TestSuperType]
- * (UNDEFINED) multiple
[oak:TestTypeA] > oak:TestSuperType
- * (UNDEFINED) multiple
[oak:TestTypeB] > oak:TestSuperType, oak:TestMixA
- * (UNDEFINED) multiple
{noformat}
If you enable {{nodeTypeIndex}} to true for say {{oak:TestMixA}} then that
would create 2 internal indexRules for {{oak:TestMixA}} and {{oak:TestTypeB}}
(where second is clone of first). So to handle OAK-4653 what we can do is just
index the indexRule name and not read primaryType and mixinTypes property values
> Nodetype index support in Lucene Index
> --
>
> Key: OAK-6831
> URL: https://issues.apache.org/jira/browse/OAK-6831
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> Lucene indexes currently support nodetype index in some form by specifying a
> property definition for "jcr:primaryType" with propertyIndex=true. However
> this can cause issue if such rules are mixed with other rules.
> For supporting usecase where same lucene index supports multiple nodetype
> rules and can be used as pure nodetype index we should have a explicit
> support for indexing nodetypes.
> *Proposal*
> Any indexRule would support following properties
> * {{nodeTypeIndex}} - Boolean indicating if this rule is for nodetype indexing
> * {{sync}} - If nodetype index is sync
> if {{nodeTypeIndex}} is enabled then any explicit property definition would
> be ignored. With this mode following index definition would be safe to use
> {noformat}
> /oak:index/nodeTypeLucene
> - jcr:primaryType = "oak:QueryIndexDefinition"
> - compatVersion = 2
> - type = "lucene"
> - async = "async"
> + indexRules
> + nt:file
> - nodeTypeIndex = true
> + app:Component
> - nodeTypeIndex = true
> {noformat}
> Here the rule order would not cause any affect as for any matching rule the
> nodes primary and mixin types would be indexed
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6831) Nodetype index support in Lucene Index
[
https://issues.apache.org/jira/browse/OAK-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205000#comment-16205000
]
Chetan Mehrotra commented on OAK-6831:
--
bq. I'm not very comfortable with the node type definition. I'd much rather
prefer that we bring in a new index type - internally we can, of course, map it
to the IndexDefinition you described above.
That would be a bigger change and something which I feel we do not require now.
What about we move this {{nodeTypeIndex}} to top level and not at individual
rule level. That would allow us to ensure that no other rule defines property
definitions. So no way to get into ambiguous state
bq. is that it would most likely index data that is not expected in the index
(e.g. OAK-4653 - what nodetype index does). Basically, we should work with all
rules instead of what getApplicableIndexingRule reports.
Note that with current design for following nodetypes
{noformat}
[oak:TestMixA]
mixin
[oak:TestSuperType]
- * (UNDEFINED) multiple
[oak:TestTypeA] > oak:TestSuperType
- * (UNDEFINED) multiple
[oak:TestTypeB] > oak:TestSuperType, oak:TestMixA
- * (UNDEFINED) multiple
{noformat}
If you enable {{nodeTypeIndex}} to true for say {{oak:TestMixA}} then that
would create 2 internal indexRules for {{oak:TestMixA}} and {{oak:TestTypeB}}
(where second is clone of first). So to handle OAK-4653 what we can do is just
index the indexRule name and not read primaryType and mixinTypes property values
> Nodetype index support in Lucene Index
> --
>
> Key: OAK-6831
> URL: https://issues.apache.org/jira/browse/OAK-6831
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> Lucene indexes currently support nodetype index in some form by specifying a
> property definition for "jcr:primaryType" with propertyIndex=true. However
> this can cause issue if such rules are mixed with other rules.
> For supporting usecase where same lucene index supports multiple nodetype
> rules and can be used as pure nodetype index we should have a explicit
> support for indexing nodetypes.
> *Proposal*
> Any indexRule would support following properties
> * {{nodeTypeIndex}} - Boolean indicating if this rule is for nodetype indexing
> * {{sync}} - If nodetype index is sync
> if {{nodeTypeIndex}} is enabled then any explicit property definition would
> be ignored. With this mode following index definition would be safe to use
> {noformat}
> /oak:index/nodeTypeLucene
> - jcr:primaryType = "oak:QueryIndexDefinition"
> - compatVersion = 2
> - type = "lucene"
> - async = "async"
> + indexRules
> + nt:file
> - nodeTypeIndex = true
> + app:Component
> - nodeTypeIndex = true
> {noformat}
> Here the rule order would not cause any affect as for any matching rule the
> nodes primary and mixin types would be indexed
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204997#comment-16204997
]
Chetan Mehrotra commented on OAK-6820:
--
bq. while validating existence of superceded index def with declaredNodeTypes
and not finding type after @ in declaredNodeTypes - maybe we should warn
That was my initial thought. But then we would need to have different index
definitions for upgrade and non upgrade case. As for fresh setups those
nodetype entries would anyway not be there
bq. there could be a case where we only update the def to supercede an (async)
index def but not reindex yet - that, afaics, would lead to disabling
superseded defs in next cycle (as reindex flag wouldn've have changed in the
cycle)
Good point. What about we change the logic slightly.
# IndexUpdate would set a hidden flag {{:disableIndexesOnNextCycle}} if it
finds during reindexing
## that current index has {{supersedes}} specified
## And those supersedes refer to indexPaths which have yet not be disabled
# IndexDisabling logic would then only be activated if that flag is present.
And then we do not require managing {{disabledOlderIndexes}}. Post disabling it
would remove that hidden flag
This approach would ensure that indexes are only and only disabled post reindex
as such a hidden flag can only be set by system after due validation
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
> Attachments: OAK-6820-v1.patch
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204997#comment-16204997
]
Chetan Mehrotra edited comment on OAK-6820 at 10/15/17 2:55 AM:
bq. while validating existence of superceded index def with declaredNodeTypes
and not finding type after @ in declaredNodeTypes - maybe we should warn
That was my initial thought. But then we would need to have different index
definitions for upgrade and non upgrade case. As for fresh setups those
nodetype entries would anyway not be there
bq. there could be a case where we only update the def to supercede an (async)
index def but not reindex yet - that, afaics, would lead to disabling
superseded defs in next cycle (as reindex flag wouldn've have changed in the
cycle)
Good point. What about we change the logic slightly.
# IndexUpdate would set a hidden flag {{:disableIndexesOnNextCycle}} if it
finds during reindexing
## that current index has {{supersedes}} specified
## And those supersedes refer to indexPaths which have yet not be disabled
# IndexDisabling logic would then only be activated if that flag is present.
And then we do not require managing {{disabledOlderIndexes}}. Post disabling it
would remove that hidden flag
This approach would ensure that indexes are only and only disabled post reindex
as such a hidden flag can only be set by system after due validation
Thoughts?
was (Author: chetanm):
bq. while validating existence of superceded index def with declaredNodeTypes
and not finding type after @ in declaredNodeTypes - maybe we should warn
That was my initial thought. But then we would need to have different index
definitions for upgrade and non upgrade case. As for fresh setups those
nodetype entries would anyway not be there
bq. there could be a case where we only update the def to supercede an (async)
index def but not reindex yet - that, afaics, would lead to disabling
superseded defs in next cycle (as reindex flag wouldn've have changed in the
cycle)
Good point. What about we change the logic slightly.
# IndexUpdate would set a hidden flag {{:disableIndexesOnNextCycle}} if it
finds during reindexing
## that current index has {{supersedes}} specified
## And those supersedes refer to indexPaths which have yet not be disabled
# IndexDisabling logic would then only be activated if that flag is present.
And then we do not require managing {{disabledOlderIndexes}}. Post disabling it
would remove that hidden flag
This approach would ensure that indexes are only and only disabled post reindex
as such a hidden flag can only be set by system after due validation
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
> Attachments: OAK-6820-v1.patch
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6831) Nodetype index support in Lucene Index
[
https://issues.apache.org/jira/browse/OAK-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204722#comment-16204722
]
Chetan Mehrotra commented on OAK-6831:
--
Some nodes on nodetype related queries
A mixin rule implies 2 modes
* mixin directly present on node - Index the jcr:mixins property
* mixin type is inherited by primary time - Index the jcr:primaryType of the
node
The query on a mixin would translate in a OR query for the actual mixin and
inherited types
> Nodetype index support in Lucene Index
> --
>
> Key: OAK-6831
> URL: https://issues.apache.org/jira/browse/OAK-6831
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> Lucene indexes currently support nodetype index in some form by specifying a
> property definition for "jcr:primaryType" with propertyIndex=true. However
> this can cause issue if such rules are mixed with other rules.
> For supporting usecase where same lucene index supports multiple nodetype
> rules and can be used as pure nodetype index we should have a explicit
> support for indexing nodetypes.
> *Proposal*
> Any indexRule would support following properties
> * {{nodeTypeIndex}} - Boolean indicating if this rule is for nodetype indexing
> * {{sync}} - If nodetype index is sync
> if {{nodeTypeIndex}} is enabled then any explicit property definition would
> be ignored. With this mode following index definition would be safe to use
> {noformat}
> /oak:index/nodeTypeLucene
> - jcr:primaryType = "oak:QueryIndexDefinition"
> - compatVersion = 2
> - type = "lucene"
> - async = "async"
> + indexRules
> + nt:file
> - nodeTypeIndex = true
> + app:Component
> - nodeTypeIndex = true
> {noformat}
> Here the rule order would not cause any affect as for any matching rule the
> nodes primary and mixin types would be indexed
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6832) Synchronous nodetype lucene index support
Chetan Mehrotra created OAK-6832: Summary: Synchronous nodetype lucene index support Key: OAK-6832 URL: https://issues.apache.org/jira/browse/OAK-6832 Project: Jackrabbit Oak Issue Type: Technical task Components: lucene Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Fix For: 1.8 Implement support for synchronous nodetype indexes. This would be built on top of support added in OAK-6831 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-6831) Nodetype index support in Lucene Index
Chetan Mehrotra created OAK-6831:
Summary: Nodetype index support in Lucene Index
Key: OAK-6831
URL: https://issues.apache.org/jira/browse/OAK-6831
Project: Jackrabbit Oak
Issue Type: Improvement
Components: lucene
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Fix For: 1.8
Lucene indexes currently support nodetype index in some form by specifying a
property definition for "jcr:primaryType" with propertyIndex=true. However this
can cause issue if such rules are mixed with other rules.
For supporting usecase where same lucene index supports multiple nodetype rules
and can be used as pure nodetype index we should have a explicit support for
indexing nodetypes.
*Proposal*
Any indexRule would support following properties
* {{nodeTypeIndex}} - Boolean indicating if this rule is for nodetype indexing
* {{sync}} - If nodetype index is sync
if {{nodeTypeIndex}} is enabled then any explicit property definition would be
ignored. With this mode following index definition would be safe to use
{noformat}
/oak:index/nodeTypeLucene
- jcr:primaryType = "oak:QueryIndexDefinition"
- compatVersion = 2
- type = "lucene"
- async = "async"
+ indexRules
+ nt:file
- nodeTypeIndex = true
+ app:Component
- nodeTypeIndex = true
{noformat}
Here the rule order would not cause any affect as for any matching rule the
nodes primary and mixin types would be indexed
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-6820:
-
Attachment: OAK-6820-v1.patch
[patch|^OAK-6820-v1.patch] for the same
[~catholicon] Please review
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
> Attachments: OAK-6820-v1.patch
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6825) oak-examples/standalone test failure on Java 9
[
https://issues.apache.org/jira/browse/OAK-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-6825.
--
Resolution: Fixed
Fix Version/s: 1.7.10
1.8
Fixed via OAK-6113
> oak-examples/standalone test failure on Java 9
> --
>
> Key: OAK-6825
> URL: https://issues.apache.org/jira/browse/OAK-6825
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: examples
>Reporter: Julian Reschke
>Assignee: Chetan Mehrotra
> Fix For: 1.8, 1.7.10
>
>
> {noformat}
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.985 sec <<<
> FAILURE! - in org.apache.jackrabbit.oak.standalone.RepositoryBootIT
> repositoryLogin(org.apache.jackrabbit.oak.standalone.RepositoryBootIT) Time
> elapsed: 0.008 sec <<< ERROR!
> java.lang.IllegalStateException: Failed to load ApplicationContext
> Caused by: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name
> 'org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration':
> Injection of autowired dependencies failed; nested exception is
> org.springframework.beans.factory.BeanCreationException: Could not autowire
> field: private org.springframework.boot.autoconfigure.mongo.MongoProperties
> org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration.properties;
> nested exception is org.springframework.beans.factory.BeanCreationException:
> Error creating bean with name 'spring.data.mongodb.CONFIGURATION_PROPERTIES':
> Initialization of bean failed; nested exception is
> java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: org.springframework.beans.factory.BeanCreationException: Could not
> autowire field: private
> org.springframework.boot.autoconfigure.mongo.MongoProperties
> org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration.properties;
> nested exception is org.springframework.beans.factory.BeanCreationException:
> Error creating bean with name 'spring.data.mongodb.CONFIGURATION_PROPERTIES':
> Initialization of bean failed; nested exception is
> java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'spring.data.mongodb.CONFIGURATION_PROPERTIES':
> Initialization of bean failed; nested exception is
> java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: java.lang.ClassNotFoundException:
> javax.xml.bind.ValidationException
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6828) IndexDefinitionBuilder should provide a way to control ordering of indexRules
Chetan Mehrotra created OAK-6828: Summary: IndexDefinitionBuilder should provide a way to control ordering of indexRules Key: OAK-6828 URL: https://issues.apache.org/jira/browse/OAK-6828 Project: Jackrabbit Oak Issue Type: Technical task Components: lucene Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Priority: Minor Fix For: 1.8 Currently the indexRules are ordered in order of calls. For better control it should provide a way where caller can influence the ordering -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (OAK-6113) update spring to 1.5.x release
[ https://issues.apache.org/jira/browse/OAK-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-6113. -- Resolution: Fixed Fix Version/s: 1.7.10 > update spring to 1.5.x release > -- > > Key: OAK-6113 > URL: https://issues.apache.org/jira/browse/OAK-6113 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: examples >Reporter: Julian Reschke >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.8, 1.7.10 > > > To 1.5.2.RELEASE or newer... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6113) update spring to 1.5.x release
[ https://issues.apache.org/jira/browse/OAK-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-6113: - Summary: update spring to 1.5.x release (was: update spring-boot-maven-plugin) > update spring to 1.5.x release > -- > > Key: OAK-6113 > URL: https://issues.apache.org/jira/browse/OAK-6113 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: examples >Reporter: Julian Reschke >Priority: Minor > Fix For: 1.8 > > > To 1.5.2.RELEASE or newer... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6825) oak-examples/standalone test failure on Java 9
[
https://issues.apache.org/jira/browse/OAK-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203275#comment-16203275
]
Chetan Mehrotra commented on OAK-6825:
--
[~julian.resc...@gmx.de] I have updated Spring dependency to latest (OAK-6113)
and now build passes on Java 9 also. Can you confirm it at your end
> oak-examples/standalone test failure on Java 9
> --
>
> Key: OAK-6825
> URL: https://issues.apache.org/jira/browse/OAK-6825
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: examples
>Reporter: Julian Reschke
>Assignee: Chetan Mehrotra
>
> {noformat}
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.985 sec <<<
> FAILURE! - in org.apache.jackrabbit.oak.standalone.RepositoryBootIT
> repositoryLogin(org.apache.jackrabbit.oak.standalone.RepositoryBootIT) Time
> elapsed: 0.008 sec <<< ERROR!
> java.lang.IllegalStateException: Failed to load ApplicationContext
> Caused by: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name
> 'org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration':
> Injection of autowired dependencies failed; nested exception is
> org.springframework.beans.factory.BeanCreationException: Could not autowire
> field: private org.springframework.boot.autoconfigure.mongo.MongoProperties
> org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration.properties;
> nested exception is org.springframework.beans.factory.BeanCreationException:
> Error creating bean with name 'spring.data.mongodb.CONFIGURATION_PROPERTIES':
> Initialization of bean failed; nested exception is
> java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: org.springframework.beans.factory.BeanCreationException: Could not
> autowire field: private
> org.springframework.boot.autoconfigure.mongo.MongoProperties
> org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration.properties;
> nested exception is org.springframework.beans.factory.BeanCreationException:
> Error creating bean with name 'spring.data.mongodb.CONFIGURATION_PROPERTIES':
> Initialization of bean failed; nested exception is
> java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'spring.data.mongodb.CONFIGURATION_PROPERTIES':
> Initialization of bean failed; nested exception is
> java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: java.lang.ClassNotFoundException:
> javax.xml.bind.ValidationException
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6113) update spring to 1.5.x release
[ https://issues.apache.org/jira/browse/OAK-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203274#comment-16203274 ] Chetan Mehrotra commented on OAK-6113: -- Done with 1812086. Had to restrict failsafe plugin too 1.18.1 due to [1] and [2]. Classpath workaround somehow not worked. For now with 1.18.1 it works fine even on Java 9 [1] https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-1.4-Release-Notes#integration-tests-with-the-maven-failsafe-plugin [2] https://github.com/spring-projects/spring-boot/issues/6254#issuecomment-281404852 > update spring to 1.5.x release > -- > > Key: OAK-6113 > URL: https://issues.apache.org/jira/browse/OAK-6113 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: examples >Reporter: Julian Reschke >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.8, 1.7.10 > > > To 1.5.2.RELEASE or newer... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (OAK-6113) update spring to 1.5.x release
[ https://issues.apache.org/jira/browse/OAK-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra reassigned OAK-6113: Assignee: Chetan Mehrotra > update spring to 1.5.x release > -- > > Key: OAK-6113 > URL: https://issues.apache.org/jira/browse/OAK-6113 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: examples >Reporter: Julian Reschke >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.8 > > > To 1.5.2.RELEASE or newer... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203107#comment-16203107
]
Chetan Mehrotra commented on OAK-6820:
--
While doing this we would also need to support disabling entries in nodetype
index like removing specific nodetype from
/oak:index/nodetype/@declaringNodeType
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6825) oak-examples/standalone test failure on Java 9
[
https://issues.apache.org/jira/browse/OAK-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203048#comment-16203048
]
Chetan Mehrotra commented on OAK-6825:
--
Update of Spring is bit tricky due some changes in boot jar model. Would have a
look at it in sometime
> oak-examples/standalone test failure on Java 9
> --
>
> Key: OAK-6825
> URL: https://issues.apache.org/jira/browse/OAK-6825
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: examples
>Reporter: Julian Reschke
>Assignee: Chetan Mehrotra
>
> {noformat}
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.985 sec <<<
> FAILURE! - in org.apache.jackrabbit.oak.standalone.RepositoryBootIT
> repositoryLogin(org.apache.jackrabbit.oak.standalone.RepositoryBootIT) Time
> elapsed: 0.008 sec <<< ERROR!
> java.lang.IllegalStateException: Failed to load ApplicationContext
> Caused by: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name
> 'org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration':
> Injection of autowired dependencies failed; nested exception is
> org.springframework.beans.factory.BeanCreationException: Could not autowire
> field: private org.springframework.boot.autoconfigure.mongo.MongoProperties
> org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration.properties;
> nested exception is org.springframework.beans.factory.BeanCreationException:
> Error creating bean with name 'spring.data.mongodb.CONFIGURATION_PROPERTIES':
> Initialization of bean failed; nested exception is
> java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: org.springframework.beans.factory.BeanCreationException: Could not
> autowire field: private
> org.springframework.boot.autoconfigure.mongo.MongoProperties
> org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration.properties;
> nested exception is org.springframework.beans.factory.BeanCreationException:
> Error creating bean with name 'spring.data.mongodb.CONFIGURATION_PROPERTIES':
> Initialization of bean failed; nested exception is
> java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'spring.data.mongodb.CONFIGURATION_PROPERTIES':
> Initialization of bean failed; nested exception is
> java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: java.lang.ClassNotFoundException:
> javax.xml.bind.ValidationException
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (OAK-6825) oak-examples/standalone test failure on Java 9
[
https://issues.apache.org/jira/browse/OAK-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra reassigned OAK-6825:
Assignee: Chetan Mehrotra
> oak-examples/standalone test failure on Java 9
> --
>
> Key: OAK-6825
> URL: https://issues.apache.org/jira/browse/OAK-6825
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: examples
>Reporter: Julian Reschke
>Assignee: Chetan Mehrotra
>
> {noformat}
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.985 sec <<<
> FAILURE! - in org.apache.jackrabbit.oak.standalone.RepositoryBootIT
> repositoryLogin(org.apache.jackrabbit.oak.standalone.RepositoryBootIT) Time
> elapsed: 0.008 sec <<< ERROR!
> java.lang.IllegalStateException: Failed to load ApplicationContext
> Caused by: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name
> 'org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration':
> Injection of autowired dependencies failed; nested exception is
> org.springframework.beans.factory.BeanCreationException: Could not autowire
> field: private org.springframework.boot.autoconfigure.mongo.MongoProperties
> org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration.properties;
> nested exception is org.springframework.beans.factory.BeanCreationException:
> Error creating bean with name 'spring.data.mongodb.CONFIGURATION_PROPERTIES':
> Initialization of bean failed; nested exception is
> java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: org.springframework.beans.factory.BeanCreationException: Could not
> autowire field: private
> org.springframework.boot.autoconfigure.mongo.MongoProperties
> org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration.properties;
> nested exception is org.springframework.beans.factory.BeanCreationException:
> Error creating bean with name 'spring.data.mongodb.CONFIGURATION_PROPERTIES':
> Initialization of bean failed; nested exception is
> java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'spring.data.mongodb.CONFIGURATION_PROPERTIES':
> Initialization of bean failed; nested exception is
> java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: java.lang.ClassNotFoundException:
> javax.xml.bind.ValidationException
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (OAK-6825) oak-examples/standalone test failure on Java 9
[
https://issues.apache.org/jira/browse/OAK-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203048#comment-16203048
]
Chetan Mehrotra edited comment on OAK-6825 at 10/13/17 4:49 AM:
Update of Spring is bit tricky due some changes in boot jar model [1]. Would
have a look at it in sometime
[1]
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-pojosr/src/main/java/org/apache/jackrabbit/oak/run/osgi/SpringBootSupport.java
was (Author: chetanm):
Update of Spring is bit tricky due some changes in boot jar model. Would have a
look at it in sometime
> oak-examples/standalone test failure on Java 9
> --
>
> Key: OAK-6825
> URL: https://issues.apache.org/jira/browse/OAK-6825
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: examples
>Reporter: Julian Reschke
>Assignee: Chetan Mehrotra
>
> {noformat}
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.985 sec <<<
> FAILURE! - in org.apache.jackrabbit.oak.standalone.RepositoryBootIT
> repositoryLogin(org.apache.jackrabbit.oak.standalone.RepositoryBootIT) Time
> elapsed: 0.008 sec <<< ERROR!
> java.lang.IllegalStateException: Failed to load ApplicationContext
> Caused by: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name
> 'org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration':
> Injection of autowired dependencies failed; nested exception is
> org.springframework.beans.factory.BeanCreationException: Could not autowire
> field: private org.springframework.boot.autoconfigure.mongo.MongoProperties
> org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration.properties;
> nested exception is org.springframework.beans.factory.BeanCreationException:
> Error creating bean with name 'spring.data.mongodb.CONFIGURATION_PROPERTIES':
> Initialization of bean failed; nested exception is
> java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: org.springframework.beans.factory.BeanCreationException: Could not
> autowire field: private
> org.springframework.boot.autoconfigure.mongo.MongoProperties
> org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration.properties;
> nested exception is org.springframework.beans.factory.BeanCreationException:
> Error creating bean with name 'spring.data.mongodb.CONFIGURATION_PROPERTIES':
> Initialization of bean failed; nested exception is
> java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'spring.data.mongodb.CONFIGURATION_PROPERTIES':
> Initialization of bean failed; nested exception is
> java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: java.lang.NoClassDefFoundError: javax/xml/bind/ValidationException
> Caused by: java.lang.ClassNotFoundException:
> javax.xml.bind.ValidationException
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6810) oak-pojosr tests fail with java 9
[
https://issues.apache.org/jira/browse/OAK-6810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202175#comment-16202175
]
Chetan Mehrotra commented on OAK-6810:
--
[~julian.resc...@gmx.de] Just fixed it by 1811991. Build now passes locally.
Can you confirm once at your end
> oak-pojosr tests fail with java 9
> -
>
> Key: OAK-6810
> URL: https://issues.apache.org/jira/browse/OAK-6810
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: pojosr
>Reporter: Julian Reschke
>Assignee: Chetan Mehrotra
>
> {noformat}
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO] Running org.apache.jackrabbit.oak.run.osgi.ConfigTest
> [ERROR] Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.975
> s <<< FAILURE! - in org.apache.jackrabbit.oak.run.osgi.ConfigTest
> [ERROR] testFileConfig(org.apache.jackrabbit.oak.run.osgi.ConfigTest) Time
> elapsed: 0.057 s <<< ERROR!
> java.lang.ClassCastException: [B cannot be cast to [C
> at
> org.apache.jackrabbit.oak.run.osgi.ConfigTest.testFileConfig(ConfigTest.groovy:73)
> [INFO] Running org.apache.jackrabbit.oak.run.osgi.DocumentNodeStoreConfigTest
> [WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
> 31.133 s - in org.apache.jackrabbit.oak.run.osgi.DocumentNodeStoreConfigTest
> [INFO] Running org.apache.jackrabbit.oak.run.osgi.HybridIndexDisabledTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.309
> s - in org.apache.jackrabbit.oak.run.osgi.HybridIndexDisabledTest
> [INFO] Running org.apache.jackrabbit.oak.run.osgi.JaasConfigSpiTest
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.342
> s <<< FAILURE! - in org.apache.jackrabbit.oak.run.osgi.JaasConfigSpiTest
> [ERROR]
> defaultConfigSpiAuth(org.apache.jackrabbit.oak.run.osgi.JaasConfigSpiTest)
> Time elapsed: 3.342 s <<< ERROR!
> java.lang.reflect.UndeclaredThrowableException
> at
> org.apache.jackrabbit.oak.run.osgi.JaasConfigSpiTest.defaultConfigSpiAuth(JaasConfigSpiTest.groovy:79)
> Caused by: java.lang.reflect.InvocationTargetException
> at
> org.apache.jackrabbit.oak.run.osgi.JaasConfigSpiTest.defaultConfigSpiAuth(JaasConfigSpiTest.groovy:79)
> Caused by: java.lang.IncompatibleClassChangeError: Method
> java.util.Set.of(Ljava/lang/Object;)Ljava/util/Set; must be
> InterfaceMethodref constant
> at
> org.apache.jackrabbit.oak.run.osgi.JaasConfigSpiTest.defaultConfigSpiAuth(JaasConfigSpiTest.groovy:79)
> [INFO] Running org.apache.jackrabbit.oak.run.osgi.JsonConfigRepFactoryTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.298
> s - in org.apache.jackrabbit.oak.run.osgi.JsonConfigRepFactoryTest
> [INFO] Running org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.041
> s - in org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
> [INFO] Running org.apache.jackrabbit.oak.run.osgi.MBeanIntegrationTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.275
> s - in org.apache.jackrabbit.oak.run.osgi.MBeanIntegrationTest
> [INFO] Running org.apache.jackrabbit.oak.run.osgi.NodeStoreConfigTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.019
> s - in org.apache.jackrabbit.oak.run.osgi.NodeStoreConfigTest
> [INFO] Running org.apache.jackrabbit.oak.run.osgi.OakOSGiRepositoryFactoryTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.345
> s - in org.apache.jackrabbit.oak.run.osgi.OakOSGiRepositoryFactoryTest
> [INFO] Running org.apache.jackrabbit.oak.run.osgi.PropertyIndexReindexingTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.33 s
> - in org.apache.jackrabbit.oak.run.osgi.PropertyIndexReindexingTest
> [INFO] Running org.apache.jackrabbit.oak.run.osgi.RepositoryClosedTest
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.211
> s - in org.apache.jackrabbit.oak.run.osgi.RepositoryClosedTest
> [INFO] Running org.apache.jackrabbit.oak.run.osgi.RepositoryShutdownTest
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.802
> s - in org.apache.jackrabbit.oak.run.osgi.RepositoryShutdownTest
> [INFO] Running
> org.apache.jackrabbit.oak.run.osgi.SecurityProviderRegistrationTest
> [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 30.563 s - in
> org.apache.jackrabbit.oak.run.osgi.SecurityProviderRegistrationTest
> [INFO] Running org.apache.jackrabbit.oak.run.osgi.SegmentNodeStoreConfigTest
> [ERROR] Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 8.039
> s <<< FAILURE! - in
>
[jira] [Commented] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
[
https://issues.apache.org/jira/browse/OAK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201854#comment-16201854
]
Chetan Mehrotra commented on OAK-6820:
--
Discussed this with [~catholicon] and we can do following
# On newer index definition set a mvp property {{supercedes}} which refers to
path of existing index definition which it replaces
{noformat}
/oak:index/assetType
- jcr:primaryType = "oak:QueryIndexDefinition"
- compatVersion = 2
- type = "lucene"
- async = "async"
- supercedes = ['/oak:index/status', '/oak:index/assetType']
{noformat}
# IndexUpdate would check for this property. If present and current index
"reindex" is false then it would check the "type" property of referred index
paths. If they are not set to "disabled" then it would set them to "disabled".
It would also set {{disabledOlderIndexes}} to true. This flag would be used to
avoid this check for later indexing cycles
# If the index replaces yet another index later then it can remove the
{{disabledOlderIndexes}} flag
[~catholicon] [~tmueller] Thoughts?
> Implement support for disabling indexes which are replaced with newer index
> ---
>
> Key: OAK-6820
> URL: https://issues.apache.org/jira/browse/OAK-6820
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: indexing
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.8
>
>
> For upgrade case in many applications older index type is set to {{disabled}}
> when new index is provisioned. If the new index is async then it would take
> some time for reindex and till then any query which used to make use of old
> index would end up traversing the repository
> To avoid such a scenario we should only mark older index as "disabled" only
> if the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6820) Implement support for disabling indexes which are replaced with newer index
Chetan Mehrotra created OAK-6820:
Summary: Implement support for disabling indexes which are
replaced with newer index
Key: OAK-6820
URL: https://issues.apache.org/jira/browse/OAK-6820
Project: Jackrabbit Oak
Issue Type: New Feature
Components: indexing
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Fix For: 1.8
For upgrade case in many applications older index type is set to {{disabled}}
when new index is provisioned. If the new index is async then it would take
some time for reindex and till then any query which used to make use of old
index would end up traversing the repository
To avoid such a scenario we should only mark older index as "disabled" only if
the newer index is reindex.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6819) Move Configuration out of DocumentNodeStoreService
[ https://issues.apache.org/jira/browse/OAK-6819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-6819. -- Resolution: Fixed Fix Version/s: 1.7.10 Done with 1811939 > Move Configuration out of DocumentNodeStoreService > -- > > Key: OAK-6819 > URL: https://issues.apache.org/jira/browse/OAK-6819 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.8, 1.7.10 > > > To simplify DocumentNodeStoreService we should move Configuration out as > first level class -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-6819) Move Configuration out of DocumentNodeStoreService
Chetan Mehrotra created OAK-6819: Summary: Move Configuration out of DocumentNodeStoreService Key: OAK-6819 URL: https://issues.apache.org/jira/browse/OAK-6819 Project: Jackrabbit Oak Issue Type: Task Components: documentmk Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Priority: Minor Fix For: 1.8 To simplify DocumentNodeStoreService we should move Configuration out as first level class -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (OAK-6803) Provide a way to for persistent cache to determine which all nodes can be cached
[
https://issues.apache.org/jira/browse/OAK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201577#comment-16201577
]
Chetan Mehrotra edited comment on OAK-6803 at 10/12/17 8:01 AM:
Applied the patch with 1811918 and 1811919
*Usage*
Set following OSGi config
{noformat}
persistentCacheIncludes=[ \
"/libs", \
"/apps", \
]
{noformat}
was (Author: chetanm):
Applied the patch with 1811918 and 1811919
> Provide a way to for persistent cache to determine which all nodes can be
> cached
>
>
> Key: OAK-6803
> URL: https://issues.apache.org/jira/browse/OAK-6803
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Labels: doc-impacting
> Fix For: 1.8, 1.7.10
>
> Attachments: OAK-6803-v1.patch, OAK-6803-v2.patch
>
>
> Currently persistent cache if enabled for nodes caches all nodes accessed on
> the system. It would be better if it can be configured to only cache those
> nodes which are not volatile so that caching can be effective
> Purpose of this issue is to
> * Provide an extension point in PersistentCache logic to check if a node is
> to be cached
> * Provide an impl which relies on some static OSGi config to determine that
> Later we can make this impl dynamic i.e. rely on access pattern to cache imp
> stuff
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6803) Provide a way to for persistent cache to determine which all nodes can be cached
[ https://issues.apache.org/jira/browse/OAK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-6803. -- Resolution: Fixed Fix Version/s: 1.7.10 > Provide a way to for persistent cache to determine which all nodes can be > cached > > > Key: OAK-6803 > URL: https://issues.apache.org/jira/browse/OAK-6803 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Minor > Labels: doc-impacting > Fix For: 1.8, 1.7.10 > > Attachments: OAK-6803-v1.patch, OAK-6803-v2.patch > > > Currently persistent cache if enabled for nodes caches all nodes accessed on > the system. It would be better if it can be configured to only cache those > nodes which are not volatile so that caching can be effective > Purpose of this issue is to > * Provide an extension point in PersistentCache logic to check if a node is > to be cached > * Provide an impl which relies on some static OSGi config to determine that > Later we can make this impl dynamic i.e. rely on access pattern to cache imp > stuff -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6803) Provide a way to for persistent cache to determine which all nodes can be cached
[ https://issues.apache.org/jira/browse/OAK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-6803: - Labels: doc-impacting (was: ) > Provide a way to for persistent cache to determine which all nodes can be > cached > > > Key: OAK-6803 > URL: https://issues.apache.org/jira/browse/OAK-6803 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Minor > Labels: doc-impacting > Fix For: 1.8 > > Attachments: OAK-6803-v1.patch, OAK-6803-v2.patch > > > Currently persistent cache if enabled for nodes caches all nodes accessed on > the system. It would be better if it can be configured to only cache those > nodes which are not volatile so that caching can be effective > Purpose of this issue is to > * Provide an extension point in PersistentCache logic to check if a node is > to be cached > * Provide an impl which relies on some static OSGi config to determine that > Later we can make this impl dynamic i.e. rely on access pattern to cache imp > stuff -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6803) Provide a way to for persistent cache to determine which all nodes can be cached
[ https://issues.apache.org/jira/browse/OAK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201577#comment-16201577 ] Chetan Mehrotra commented on OAK-6803: -- Applied the patch with 1811918 and 1811919 > Provide a way to for persistent cache to determine which all nodes can be > cached > > > Key: OAK-6803 > URL: https://issues.apache.org/jira/browse/OAK-6803 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Minor > Labels: doc-impacting > Fix For: 1.8 > > Attachments: OAK-6803-v1.patch, OAK-6803-v2.patch > > > Currently persistent cache if enabled for nodes caches all nodes accessed on > the system. It would be better if it can be configured to only cache those > nodes which are not volatile so that caching can be effective > Purpose of this issue is to > * Provide an extension point in PersistentCache logic to check if a node is > to be cached > * Provide an impl which relies on some static OSGi config to determine that > Later we can make this impl dynamic i.e. rely on access pattern to cache imp > stuff -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6787) Delete property index entries recursively in batches to avoid large transaction
[ https://issues.apache.org/jira/browse/OAK-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201529#comment-16201529 ] Chetan Mehrotra commented on OAK-6787: -- Updated the logic with r1811912 to allow batching across multiple paths which are passed as root paths > Delete property index entries recursively in batches to avoid large > transaction > --- > > Key: OAK-6787 > URL: https://issues.apache.org/jira/browse/OAK-6787 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8, 1.7.10 > > Attachments: OAK-6787-v1.patch, OAK-6787-v2.patch > > > PropertyIndexCleaner currently remove the property index bucket in a single > remove. This would work fine with SegmentNodeStore but would cause issue with > DocumentNodeStore where it would result in a one large commit. > To avoid this scenario we should implement recursive delete. This approach > would only be used if NodeStore implements Clusterable -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (OAK-6815) Support specifying queryPaths in IndexDefinitionBuilder
[ https://issues.apache.org/jira/browse/OAK-6815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-6815. -- Resolution: Fixed Fix Version/s: 1.7.10 Done with 1811911 > Support specifying queryPaths in IndexDefinitionBuilder > --- > > Key: OAK-6815 > URL: https://issues.apache.org/jira/browse/OAK-6815 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.8, 1.7.10 > > > IndexDefinitionBuilder should support "queryPaths" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-6815) Support specifying queryPaths in IndexDefinitionBuilder
Chetan Mehrotra created OAK-6815: Summary: Support specifying queryPaths in IndexDefinitionBuilder Key: OAK-6815 URL: https://issues.apache.org/jira/browse/OAK-6815 Project: Jackrabbit Oak Issue Type: Improvement Components: lucene Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Priority: Minor Fix For: 1.8 IndexDefinitionBuilder should support "queryPaths" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6803) Provide a way to for persistent cache to determine which all nodes can be cached
[
https://issues.apache.org/jira/browse/OAK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-6803:
-
Attachment: OAK-6803-v2.patch
[updated patch|^OAK-6803-v2.patch] with benchmark support
#Results without enabling this feature
{noformat}
$ java -jar oak-benchmarks-1.8-SNAPSHOT.jar benchmark SetPropertyTest Oak-Mongo
Apache Jackrabbit Oak 1.8-SNAPSHOT
# SetPropertyTest C min 10% 50% 90% max
N
Oak-Mongo 120442101245432883318
24
{noformat}
# Results post enabling this
{noformat}
java -DpersistentCacheIncludes="/libs,/apps" -jar
oak-benchmarks-1.8-SNAPSHOT.jar benchmark SetPropertyTest Oak-Mongo
Apache Jackrabbit Oak 1.8-SNAPSHOT
# SetPropertyTest C min 10% 50% 90% max
N
Configuring persistent cache to only cache nodes under paths [/libs, /apps]
Oak-Mongo 117711896211527842922
28
{noformat}
[~mreutegg] Please review
> Provide a way to for persistent cache to determine which all nodes can be
> cached
>
>
> Key: OAK-6803
> URL: https://issues.apache.org/jira/browse/OAK-6803
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8
>
> Attachments: OAK-6803-v1.patch, OAK-6803-v2.patch
>
>
> Currently persistent cache if enabled for nodes caches all nodes accessed on
> the system. It would be better if it can be configured to only cache those
> nodes which are not volatile so that caching can be effective
> Purpose of this issue is to
> * Provide an extension point in PersistentCache logic to check if a node is
> to be cached
> * Provide an impl which relies on some static OSGi config to determine that
> Later we can make this impl dynamic i.e. rely on access pattern to cache imp
> stuff
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (OAK-6803) Provide a way to for persistent cache to determine which all nodes can be cached
[
https://issues.apache.org/jira/browse/OAK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200649#comment-16200649
]
Chetan Mehrotra edited comment on OAK-6803 at 10/11/17 5:40 PM:
[updated patch|^OAK-6803-v2.patch] with benchmark support
# Results without enabling this feature
{noformat}
$ java -jar oak-benchmarks-1.8-SNAPSHOT.jar benchmark SetPropertyTest Oak-Mongo
Apache Jackrabbit Oak 1.8-SNAPSHOT
# SetPropertyTest C min 10% 50% 90% max
N
Oak-Mongo 120442101245432883318
24
{noformat}
# Results post enabling this
{noformat}
java -DpersistentCacheIncludes="/libs,/apps" -jar
oak-benchmarks-1.8-SNAPSHOT.jar benchmark SetPropertyTest Oak-Mongo
Apache Jackrabbit Oak 1.8-SNAPSHOT
# SetPropertyTest C min 10% 50% 90% max
N
Configuring persistent cache to only cache nodes under paths [/libs, /apps]
Oak-Mongo 117711896211527842922
28
{noformat}
[~mreutegg] Please review
was (Author: chetanm):
[updated patch|^OAK-6803-v2.patch] with benchmark support
#Results without enabling this feature
{noformat}
$ java -jar oak-benchmarks-1.8-SNAPSHOT.jar benchmark SetPropertyTest Oak-Mongo
Apache Jackrabbit Oak 1.8-SNAPSHOT
# SetPropertyTest C min 10% 50% 90% max
N
Oak-Mongo 120442101245432883318
24
{noformat}
# Results post enabling this
{noformat}
java -DpersistentCacheIncludes="/libs,/apps" -jar
oak-benchmarks-1.8-SNAPSHOT.jar benchmark SetPropertyTest Oak-Mongo
Apache Jackrabbit Oak 1.8-SNAPSHOT
# SetPropertyTest C min 10% 50% 90% max
N
Configuring persistent cache to only cache nodes under paths [/libs, /apps]
Oak-Mongo 117711896211527842922
28
{noformat}
[~mreutegg] Please review
> Provide a way to for persistent cache to determine which all nodes can be
> cached
>
>
> Key: OAK-6803
> URL: https://issues.apache.org/jira/browse/OAK-6803
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8
>
> Attachments: OAK-6803-v1.patch, OAK-6803-v2.patch
>
>
> Currently persistent cache if enabled for nodes caches all nodes accessed on
> the system. It would be better if it can be configured to only cache those
> nodes which are not volatile so that caching can be effective
> Purpose of this issue is to
> * Provide an extension point in PersistentCache logic to check if a node is
> to be cached
> * Provide an impl which relies on some static OSGi config to determine that
> Later we can make this impl dynamic i.e. rely on access pattern to cache imp
> stuff
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6787) Delete property index entries recursively in batches to avoid large transaction
[ https://issues.apache.org/jira/browse/OAK-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-6787. -- Resolution: Fixed Fix Version/s: 1.7.10 Done with 1811710 > Delete property index entries recursively in batches to avoid large > transaction > --- > > Key: OAK-6787 > URL: https://issues.apache.org/jira/browse/OAK-6787 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8, 1.7.10 > > Attachments: OAK-6787-v1.patch, OAK-6787-v2.patch > > > PropertyIndexCleaner currently remove the property index bucket in a single > remove. This would work fine with SegmentNodeStore but would cause issue with > DocumentNodeStore where it would result in a one large commit. > To avoid this scenario we should implement recursive delete. This approach > would only be used if NodeStore implements Clusterable -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6787) Delete property index entries recursively in batches to avoid large transaction
[ https://issues.apache.org/jira/browse/OAK-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-6787: - Attachment: OAK-6787-v2.patch [updated patch|^OAK-6787-v2.patch] > Delete property index entries recursively in batches to avoid large > transaction > --- > > Key: OAK-6787 > URL: https://issues.apache.org/jira/browse/OAK-6787 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8 > > Attachments: OAK-6787-v1.patch, OAK-6787-v2.patch > > > PropertyIndexCleaner currently remove the property index bucket in a single > remove. This would work fine with SegmentNodeStore but would cause issue with > DocumentNodeStore where it would result in a one large commit. > To avoid this scenario we should implement recursive delete. This approach > would only be used if NodeStore implements Clusterable -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6803) Provide a way to for persistent cache to determine which all nodes can be cached
[ https://issues.apache.org/jira/browse/OAK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198423#comment-16198423 ] Chetan Mehrotra commented on OAK-6803: -- Missed that. Okie would change createCachePredicate as per below # if default config is found then it would return alwaysTrue # if config array is empty then also alwaysTrue (or should we disable persistent cache then?) # if non empty and non default use a PathFilter > Provide a way to for persistent cache to determine which all nodes can be > cached > > > Key: OAK-6803 > URL: https://issues.apache.org/jira/browse/OAK-6803 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.8 > > Attachments: OAK-6803-v1.patch > > > Currently persistent cache if enabled for nodes caches all nodes accessed on > the system. It would be better if it can be configured to only cache those > nodes which are not volatile so that caching can be effective > Purpose of this issue is to > * Provide an extension point in PersistentCache logic to check if a node is > to be cached > * Provide an impl which relies on some static OSGi config to determine that > Later we can make this impl dynamic i.e. rely on access pattern to cache imp > stuff -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6787) Delete property index entries recursively in batches to avoid large transaction
[ https://issues.apache.org/jira/browse/OAK-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198372#comment-16198372 ] Chetan Mehrotra commented on OAK-6787: -- bq. What is the reason you are using nodeStore.reset(builder) instead of a more common builder = nodeStore.getRoot().builder()? Aah thats an artifact of a previous approach I was trying where I was reusing the older NodeBuilder and performing remove on that as part of traversal. Would replace that call with explicitly creation of new builder instance > Delete property index entries recursively in batches to avoid large > transaction > --- > > Key: OAK-6787 > URL: https://issues.apache.org/jira/browse/OAK-6787 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8 > > Attachments: OAK-6787-v1.patch > > > PropertyIndexCleaner currently remove the property index bucket in a single > remove. This would work fine with SegmentNodeStore but would cause issue with > DocumentNodeStore where it would result in a one large commit. > To avoid this scenario we should implement recursive delete. This approach > would only be used if NodeStore implements Clusterable -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6741) Switch to official OSGi component and metatype annotations
[ https://issues.apache.org/jira/browse/OAK-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198327#comment-16198327 ] Chetan Mehrotra commented on OAK-6741: -- bq. Last week I proposed a patch to address the issues you mentioned. It would be great if you could review the changes. I'd like to commit them soon. Patch looks ok. However note that my conclusion may not be exhaustive as I did not had time to go through all difference reported in tooling. If you can run tooling at you end and validate the diff report then that would be more confirmative validation > Switch to official OSGi component and metatype annotations > -- > > Key: OAK-6741 > URL: https://issues.apache.org/jira/browse/OAK-6741 > Project: Jackrabbit Oak > Issue Type: Improvement >Reporter: Robert Munteanu > Fix For: 1.8, 1.7.10 > > Attachments: OAK-6741-proposed-changes-chetans-feedback.patch, > osgi-metadata-1.7.8.json, osgi-metadata-trunk.json > > > We should remove the 'old' Felix SCR annotations and move to the 'new' OSGi > R6 annotations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (OAK-6790) FacetResult class isn't exposed anymore
[
https://issues.apache.org/jira/browse/OAK-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198260#comment-16198260
]
Chetan Mehrotra edited comment on OAK-6790 at 10/10/17 6:47 AM:
bq. Otoh, not picking up oak-query-spi on such setup would still fail to
resolve the bundle.
That would. But here case would be that Oak user updated to newer version of
Oak based application which adds all the new bundles. However his bundle would
still not resolve even with oak-uery-spi being present
bq. All that said, I think we have moved around quite a few packages in the
refactoring exercise. So, if we're ok with a bit of backward incompatibility
hit, I think the current place (bundle and package) is more fitting place.
Most of such moves involved packages which were to be consumed by other Oak
bundles hence changing them does not impact backward compatability. However
this package was meant to be consumer by non Oak bundles and hence would pose
problem. Note that this package was a version export [1] so something which we
say is supported for backward compatability
[1]
https://github.com/apache/jackrabbit-oak/blob/jackrabbit-oak-1.6.5/oak-core/src/main/java/org/apache/jackrabbit/oak/query/facet/package-info.java
was (Author: chetanm):
bq. Otoh, not picking up oak-query-spi on such setup would still fail to
resolve the bundle.
That would. But here case would be that Oak user updated to newer version of
Oak based application which adds all the new bundles. However his bundle would
still not resolve even with oak-uery-spi being present
bq. All that said, I think we have moved around quite a few packages in the
refactoring exercise. So, if we're ok with a bit of backward incompatibility
hit, I think the current place (bundle and package) is more fitting place.
Most of such moves involved packages which were to be consumed by other Oak
bundles hence changing them does not impact backward compatability. However
this package was meant to be consumer by non Oak bundles and hence would pose
problem
> FacetResult class isn't exposed anymore
> ---
>
> Key: OAK-6790
> URL: https://issues.apache.org/jira/browse/OAK-6790
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: query
>Affects Versions: 1.7.2
>Reporter: Vikas Saurabh
>Assignee: Vikas Saurabh
> Fix For: 1.8, 1.7.10
>
>
> OAK-3847 exposed {{FacetResult}} (rather
> {{org.apache.jackrabbit.oak.query.facet}} package) to easily consume
> extracted facet data. That exported package got removed during OAK-6304 (as
> noted in this comment \[1]).
> We need to export that class - maybe that class makes sense as part of
> {{oak-query-spi}} module - that still needs to be checked as assessed btw (ie
> if we can simply re-enable the removed export AND that its current place in
> oak-core still makes sense or not).
> [~anchela], would you remember what was the warning that was observed without
> removing this export (disclaimer: I still feel that it belongs in
> {{oak-query-spi}}... I'm just trying to have all data available).
> \[1]:
> https://issues.apache.org/jira/browse/OAK-6304?focusedCommentId=16042721=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16042721
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6790) FacetResult class isn't exposed anymore
[
https://issues.apache.org/jira/browse/OAK-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198260#comment-16198260
]
Chetan Mehrotra commented on OAK-6790:
--
bq. Otoh, not picking up oak-query-spi on such setup would still fail to
resolve the bundle.
That would. But here case would be that Oak user updated to newer version of
Oak based application which adds all the new bundles. However his bundle would
still not resolve even with oak-uery-spi being present
bq. All that said, I think we have moved around quite a few packages in the
refactoring exercise. So, if we're ok with a bit of backward incompatibility
hit, I think the current place (bundle and package) is more fitting place.
Most of such moves involved packages which were to be consumed by other Oak
bundles hence changing them does not impact backward compatability. However
this package was meant to be consumer by non Oak bundles and hence would pose
problem
> FacetResult class isn't exposed anymore
> ---
>
> Key: OAK-6790
> URL: https://issues.apache.org/jira/browse/OAK-6790
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: query
>Affects Versions: 1.7.2
>Reporter: Vikas Saurabh
>Assignee: Vikas Saurabh
> Fix For: 1.8, 1.7.10
>
>
> OAK-3847 exposed {{FacetResult}} (rather
> {{org.apache.jackrabbit.oak.query.facet}} package) to easily consume
> extracted facet data. That exported package got removed during OAK-6304 (as
> noted in this comment \[1]).
> We need to export that class - maybe that class makes sense as part of
> {{oak-query-spi}} module - that still needs to be checked as assessed btw (ie
> if we can simply re-enable the removed export AND that its current place in
> oak-core still makes sense or not).
> [~anchela], would you remember what was the warning that was observed without
> removing this export (disclaimer: I still feel that it belongs in
> {{oak-query-spi}}... I'm just trying to have all data available).
> \[1]:
> https://issues.apache.org/jira/browse/OAK-6304?focusedCommentId=16042721=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16042721
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6790) FacetResult class isn't exposed anymore
[
https://issues.apache.org/jira/browse/OAK-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198195#comment-16198195
]
Chetan Mehrotra commented on OAK-6790:
--
bq. I'm guessing consumer of this class would have to mark dependency on
oak-query-spi anyway
Not necessarily as his code would still compile fine with older maven
dependency however his existing working app would not work post upgrade as his
bundle using the older FacetResult would not resolve
> FacetResult class isn't exposed anymore
> ---
>
> Key: OAK-6790
> URL: https://issues.apache.org/jira/browse/OAK-6790
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: query
>Affects Versions: 1.7.2
>Reporter: Vikas Saurabh
>Assignee: Vikas Saurabh
> Fix For: 1.8, 1.7.10
>
>
> OAK-3847 exposed {{FacetResult}} (rather
> {{org.apache.jackrabbit.oak.query.facet}} package) to easily consume
> extracted facet data. That exported package got removed during OAK-6304 (as
> noted in this comment \[1]).
> We need to export that class - maybe that class makes sense as part of
> {{oak-query-spi}} module - that still needs to be checked as assessed btw (ie
> if we can simply re-enable the removed export AND that its current place in
> oak-core still makes sense or not).
> [~anchela], would you remember what was the warning that was observed without
> removing this export (disclaimer: I still feel that it belongs in
> {{oak-query-spi}}... I'm just trying to have all data available).
> \[1]:
> https://issues.apache.org/jira/browse/OAK-6304?focusedCommentId=16042721=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16042721
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6763) Convert oak-examples to OSGi R6 annotations
[ https://issues.apache.org/jira/browse/OAK-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-6763. -- Resolution: Fixed Assignee: Chetan Mehrotra > Convert oak-examples to OSGi R6 annotations > --- > > Key: OAK-6763 > URL: https://issues.apache.org/jira/browse/OAK-6763 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: examples >Reporter: Robert Munteanu >Assignee: Chetan Mehrotra > Fix For: 1.8, 1.7.10 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6763) Convert oak-examples to OSGi R6 annotations
[ https://issues.apache.org/jira/browse/OAK-6763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198160#comment-16198160 ] Chetan Mehrotra commented on OAK-6763: -- Modules under examples are not OSGi bundles. There was a unused dependency on Felix SCR Annotation in oak-standalone. Removed that with r1811637 > Convert oak-examples to OSGi R6 annotations > --- > > Key: OAK-6763 > URL: https://issues.apache.org/jira/browse/OAK-6763 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: examples >Reporter: Robert Munteanu > Fix For: 1.8, 1.7.10 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6803) Provide a way to for persistent cache to determine which all nodes can be cached
[ https://issues.apache.org/jira/browse/OAK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-6803: - Summary: Provide a way to for persistent cache to determine which all nodes can be cached (was: Provide a way to for persistent cache to determine which all nodes are cached) > Provide a way to for persistent cache to determine which all nodes can be > cached > > > Key: OAK-6803 > URL: https://issues.apache.org/jira/browse/OAK-6803 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.8 > > Attachments: OAK-6803-v1.patch > > > Currently persistent cache if enabled for nodes caches all nodes accessed on > the system. It would be better if it can be configured to only cache those > nodes which are not volatile so that caching can be effective > Purpose of this issue is to > * Provide an extension point in PersistentCache logic to check if a node is > to be cached > * Provide an impl which relies on some static OSGi config to determine that > Later we can make this impl dynamic i.e. rely on access pattern to cache imp > stuff -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6790) FacetResult class isn't exposed anymore
[
https://issues.apache.org/jira/browse/OAK-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198153#comment-16198153
]
Chetan Mehrotra commented on OAK-6790:
--
[~catholicon] If the package is not used in multiple bundles
"org.apache.jackrabbit.oak.query.facet" then lets keep the same package name to
avoid any backward compatibility issue as this class was sort of api
> FacetResult class isn't exposed anymore
> ---
>
> Key: OAK-6790
> URL: https://issues.apache.org/jira/browse/OAK-6790
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: query
>Affects Versions: 1.7.2
>Reporter: Vikas Saurabh
>Assignee: Vikas Saurabh
> Fix For: 1.8, 1.7.10
>
>
> OAK-3847 exposed {{FacetResult}} (rather
> {{org.apache.jackrabbit.oak.query.facet}} package) to easily consume
> extracted facet data. That exported package got removed during OAK-6304 (as
> noted in this comment \[1]).
> We need to export that class - maybe that class makes sense as part of
> {{oak-query-spi}} module - that still needs to be checked as assessed btw (ie
> if we can simply re-enable the removed export AND that its current place in
> oak-core still makes sense or not).
> [~anchela], would you remember what was the warning that was observed without
> removing this export (disclaimer: I still feel that it belongs in
> {{oak-query-spi}}... I'm just trying to have all data available).
> \[1]:
> https://issues.apache.org/jira/browse/OAK-6304?focusedCommentId=16042721=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16042721
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6803) Provide a way to for persistent cache to determine which all nodes are cached
[
https://issues.apache.org/jira/browse/OAK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-6803:
-
Attachment: OAK-6803-v1.patch
[patch|^OAK-6803-v1.patch] for the same.
It introduces a new OSGi config {{persistentCacheIncludes}} which can be used
to configure an array of paths which are (including there children) allowed to
be included in persistentCache. It defaults to "['/']"
[~tomek.rekawek] [~mreutegg] Please review
> Provide a way to for persistent cache to determine which all nodes are cached
> -
>
> Key: OAK-6803
> URL: https://issues.apache.org/jira/browse/OAK-6803
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Reporter: Chetan Mehrotra
> Fix For: 1.8
>
> Attachments: OAK-6803-v1.patch
>
>
> Currently persistent cache if enabled for nodes caches all nodes accessed on
> the system. It would be better if it can be configured to only cache those
> nodes which are not volatile so that caching can be effective
> Purpose of this issue is to
> * Provide an extension point in PersistentCache logic to check if a node is
> to be cached
> * Provide an impl which relies on some static OSGi config to determine that
> Later we can make this impl dynamic i.e. rely on access pattern to cache imp
> stuff
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (OAK-6803) Provide a way to for persistent cache to determine which all nodes are cached
[ https://issues.apache.org/jira/browse/OAK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra reassigned OAK-6803: Assignee: Chetan Mehrotra > Provide a way to for persistent cache to determine which all nodes are cached > - > > Key: OAK-6803 > URL: https://issues.apache.org/jira/browse/OAK-6803 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8 > > Attachments: OAK-6803-v1.patch > > > Currently persistent cache if enabled for nodes caches all nodes accessed on > the system. It would be better if it can be configured to only cache those > nodes which are not volatile so that caching can be effective > Purpose of this issue is to > * Provide an extension point in PersistentCache logic to check if a node is > to be cached > * Provide an impl which relies on some static OSGi config to determine that > Later we can make this impl dynamic i.e. rely on access pattern to cache imp > stuff -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6803) Provide a way to for persistent cache to determine which all nodes are cached
[ https://issues.apache.org/jira/browse/OAK-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-6803: - Priority: Minor (was: Major) > Provide a way to for persistent cache to determine which all nodes are cached > - > > Key: OAK-6803 > URL: https://issues.apache.org/jira/browse/OAK-6803 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.8 > > Attachments: OAK-6803-v1.patch > > > Currently persistent cache if enabled for nodes caches all nodes accessed on > the system. It would be better if it can be configured to only cache those > nodes which are not volatile so that caching can be effective > Purpose of this issue is to > * Provide an extension point in PersistentCache logic to check if a node is > to be cached > * Provide an impl which relies on some static OSGi config to determine that > Later we can make this impl dynamic i.e. rely on access pattern to cache imp > stuff -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-6803) Provide a way to for persistent cache to determine which all nodes are cached
Chetan Mehrotra created OAK-6803: Summary: Provide a way to for persistent cache to determine which all nodes are cached Key: OAK-6803 URL: https://issues.apache.org/jira/browse/OAK-6803 Project: Jackrabbit Oak Issue Type: Improvement Components: documentmk Reporter: Chetan Mehrotra Fix For: 1.8 Currently persistent cache if enabled for nodes caches all nodes accessed on the system. It would be better if it can be configured to only cache those nodes which are not volatile so that caching can be effective Purpose of this issue is to * Provide an extension point in PersistentCache logic to check if a node is to be cached * Provide an impl which relies on some static OSGi config to determine that Later we can make this impl dynamic i.e. rely on access pattern to cache imp stuff -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6787) Delete property index entries recursively in batches to avoid large transaction
[ https://issues.apache.org/jira/browse/OAK-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16195573#comment-16195573 ] Chetan Mehrotra commented on OAK-6787: -- bq. I guess that check would happen at the wiring end though, right? Ack. Would check in LuceneIndexProviderService and configure that in PropertyIndexCleaner at time of init. PropertyIndexCleaner would have a flag exposed for recursiveDelete which would be enabled by LuceneIndexProviderService upon determining the type of NodeStore > Delete property index entries recursively in batches to avoid large > transaction > --- > > Key: OAK-6787 > URL: https://issues.apache.org/jira/browse/OAK-6787 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8 > > Attachments: OAK-6787-v1.patch > > > PropertyIndexCleaner currently remove the property index bucket in a single > remove. This would work fine with SegmentNodeStore but would cause issue with > DocumentNodeStore where it would result in a one large commit. > To avoid this scenario we should implement recursive delete. This approach > would only be used if NodeStore implements Clusterable -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6787) Delete property index entries recursively in batches to avoid large transaction
[ https://issues.apache.org/jira/browse/OAK-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-6787: - Attachment: OAK-6787-v1.patch [patch|^OAK-6787-v1.patch] for the same (wiring to cleaner missing) [~mreutegg] [~catholicon] Please review. I was thinking to use this when NodeStore is of type Clusterable. wdyt? > Delete property index entries recursively in batches to avoid large > transaction > --- > > Key: OAK-6787 > URL: https://issues.apache.org/jira/browse/OAK-6787 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8 > > Attachments: OAK-6787-v1.patch > > > PropertyIndexCleaner currently remove the property index bucket in a single > remove. This would work fine with SegmentNodeStore but would cause issue with > DocumentNodeStore where it would result in a one large commit. > To avoid this scenario we should implement recursive delete. This approach > would only be used if NodeStore implements Clusterable -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6218) Including id in DocumentStoreException which wrap MongoException
[
https://issues.apache.org/jira/browse/OAK-6218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194224#comment-16194224
]
Chetan Mehrotra commented on OAK-6218:
--
[~mreutegg] This issue is seen quite a few times now. Would be good if we can
have this in 1.8
> Including id in DocumentStoreException which wrap MongoException
>
>
> Key: OAK-6218
> URL: https://issues.apache.org/jira/browse/OAK-6218
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: mongomk
>Reporter: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8
>
>
> Currently when any exception occurs at Mongo level it gets wrapped in
> DocumentStoreException. To help in debugging such issues it would be good to
> also include the documentId(s) which was being processed in that call as part
> of exception message
> {noformat}
> Caused by: org.apache.jackrabbit.oak.plugins.document.DocumentStoreException:
> { "serverUsed" : "mongoserver:20001" , "ok" : 1 , "n" : 0 , "updatedExisting"
> : false , "err" : "Resulting document after update is larger than 16777216" ,
> "code" : 17419}
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentStoreException.convert(DocumentStoreException.java:48)
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentStoreException.convert(DocumentStoreException.java:41)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.findAndModify(MongoDocumentStore.java:789)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.createOrUpdate(MongoDocumentStore.java:805)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.createOrUpdate(MongoDocumentStore.java:884)
> at
> org.apache.jackrabbit.oak.plugins.document.util.LeaseCheckDocumentStoreWrapper.createOrUpdate(LeaseCheckDocumentStoreWrapper.java:133)
> at
> org.apache.jackrabbit.oak.plugins.document.Commit.applyToDocumentStore(Commit.java:308)
> at
> org.apache.jackrabbit.oak.plugins.document.Commit.applyToDocumentStore(Commit.java:245)
> at
> org.apache.jackrabbit.oak.plugins.document.Commit.applyInternal(Commit.java:214)
> at
> org.apache.jackrabbit.oak.plugins.document.Commit.apply(Commit.java:203)
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.persist(DocumentNodeStoreBranch.java:292)
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.persist(DocumentNodeStoreBranch.java:262)
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.access$300(DocumentNodeStoreBranch.java:57)
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch$InMemory.merge(DocumentNodeStoreBranch.java:499)
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.merge0(DocumentNodeStoreBranch.java:182)
> ... 36 common frames omitted
> Caused by: com.mongodb.WriteConcernException: { "serverUsed" :
> "mongoserver:20001" , "ok" : 1 , "n" : 0 , "updatedExisting" : false , "err"
> : "Resulting document after update is larger than 16777216" , "code" : 17419}
> at com.mongodb.CommandResult.getWriteException(CommandResult.java:90)
> at com.mongodb.CommandResult.getException(CommandResult.java:79)
> at
> com.mongodb.DBCollectionImpl.translateBulkWriteException(DBCollectionImpl.java:414)
> at com.mongodb.DBCollectionImpl.updateImpl(DBCollectionImpl.java:292)
> at com.mongodb.DBCollection.update(DBCollection.java:250)
> at com.mongodb.DBCollection.update(DBCollection.java:232)
> at com.mongodb.DBCollection.update(DBCollection.java:307)
> at com.mongodb.DBCollection.update(DBCollection.java:322)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.findAndModify(MongoDocumentStore.java:746)
> ... 48 common frames omitted
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6787) Delete property index entries recursively in batches to avoid large transaction
Chetan Mehrotra created OAK-6787: Summary: Delete property index entries recursively in batches to avoid large transaction Key: OAK-6787 URL: https://issues.apache.org/jira/browse/OAK-6787 Project: Jackrabbit Oak Issue Type: Technical task Components: lucene Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Fix For: 1.8 PropertyIndexCleaner currently remove the property index bucket in a single remove. This would work fine with SegmentNodeStore but would cause issue with DocumentNodeStore where it would result in a one large commit. To avoid this scenario we should implement recursive delete. This approach would only be used if NodeStore implements Clusterable -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6741) Switch to official OSGi component and metatype annotations
[
https://issues.apache.org/jira/browse/OAK-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-6741:
-
Attachment: osgi-metadata-1.7.8.json
osgi-metadata-trunk.json
I tried to implement a hacky script [1] to generate report for the DS and
Metatype xml as json.
The script can be used like below (requires Groovy installed)
{noformat}
groovy
https://gist.githubusercontent.com/chetanmeh/3cf69a690e67f9ceb9c1abcbfadc495b/raw/analyzeOsgiMetadata.groovy
/path/to/jackrabbit-oak
{noformat}
The script would search for all jar files and would then extract the metatype
and ds xml and convert them into sorted json. This json can then be compared
with a diff tool
See attached [trunk json|^osgi-metadata-trunk.json] and [Oak 1.7.8 tag
json|^osgi-metadata-1.7.8.json]
Diff show some false positive so need to be checked once manually. For now it
reports following issues (listing some)
* No 'Designate' found in
OSGI-INF/metatype/org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl$Configuration.xml
- Looks like AuthorizationConfigurationImpl does not have designate specified
* COWNodeStoreService is having the "role" property missing. Which is required
* DocumentNodeStoreService - Missing props
** updateLimit
** repository.home
** role
* SecondaryStoreCacheService - immediate missing
* MountInfoProviderService - supportFragment missing (may be it was changed so
valid)
* AuthenticationConfigurationImpl
** org.apache.jackrabbit.oak.authentication.appName missing
** org.apache.jackrabbit.oak.authentication.configSpiName missing
* DefaultAuthorizableActionProvider - label and value swapped
There are few I may have missed from quick look at diff. So have a look again.
Hope this would be useful in this migration effort
[1] https://gist.github.com/chetanmeh/3cf69a690e67f9ceb9c1abcbfadc495b
> Switch to official OSGi component and metatype annotations
> --
>
> Key: OAK-6741
> URL: https://issues.apache.org/jira/browse/OAK-6741
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Robert Munteanu
> Fix For: 1.8, 1.7.9
>
> Attachments: osgi-metadata-1.7.8.json, osgi-metadata-trunk.json
>
>
> We should remove the 'old' Felix SCR annotations and move to the 'new' OSGi
> R6 annotations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
