[jira] [Resolved] (OAK-3068) Lucene IndexCopier improved logs around the CopyOnRead feature
[ https://issues.apache.org/jira/browse/OAK-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu resolved OAK-3068. -- Resolution: Duplicate Fix Version/s: (was: 1.3.3) (was: 1.2.3) marking as duplicate of OAK-3069 > Lucene IndexCopier improved logs around the CopyOnRead feature > -- > > Key: OAK-3068 > URL: https://issues.apache.org/jira/browse/OAK-3068 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: lucene >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Attachments: OAK-3068-v0.patch > > > I'd like to add some more logs related to the remove vs local file read > operations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3069) Provide option to eagerly copy the new index files in CopyOnRead
[ https://issues.apache.org/jira/browse/OAK-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14613165#comment-14613165 ] Alex Parvulescu commented on OAK-3069: -- thanks for taking care of this one! I added a few minor tweaks that I noticed since the last patch : http://svn.apache.org/r1689008 > Provide option to eagerly copy the new index files in CopyOnRead > > > Key: OAK-3069 > URL: https://issues.apache.org/jira/browse/OAK-3069 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Labels: docs-impacting > Fix For: 1.2.3, 1.3.3, 1.0.17 > > Attachments: OAK-3069+logs-v2.patch, OAK-3069+logs.patch, > OAK-3069.patch > > > Currently the CopyOnRead feature lazily starts copying the index files upon > first access. This at times does not work efficiently because the index gets > updated frequently and hence by the time copy is done index would have moved > on. Due to this most of the index read turn out to be remote reads and cause > slowness > We should provide an option on per index basis to enable copying the files > eagerly before exposing the new index to the query engine such that any read > done is done locally -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3069) Provide option to eagerly copy the new index files in CopyOnRead
[ https://issues.apache.org/jira/browse/OAK-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-3069: - Labels: docs-impacting (was: ) > Provide option to eagerly copy the new index files in CopyOnRead > > > Key: OAK-3069 > URL: https://issues.apache.org/jira/browse/OAK-3069 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Labels: docs-impacting > Fix For: 1.2.3, 1.3.3, 1.0.17 > > Attachments: OAK-3069+logs-v2.patch, OAK-3069+logs.patch, > OAK-3069.patch > > > Currently the CopyOnRead feature lazily starts copying the index files upon > first access. This at times does not work efficiently because the index gets > updated frequently and hence by the time copy is done index would have moved > on. Due to this most of the index read turn out to be remote reads and cause > slowness > We should provide an option on per index basis to enable copying the files > eagerly before exposing the new index to the query engine such that any read > done is done locally -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3068) Lucene IndexCopier improved logs around the CopyOnRead feature
[
https://issues.apache.org/jira/browse/OAK-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14613151#comment-14613151
]
Chetan Mehrotra commented on OAK-3068:
--
Applied the patch affecting {{LuceneIndexEditor}} in
http://svn.apache.org/r1689004
> Lucene IndexCopier improved logs around the CopyOnRead feature
> --
>
> Key: OAK-3068
> URL: https://issues.apache.org/jira/browse/OAK-3068
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
> Fix For: 1.2.3, 1.3.3
>
> Attachments: OAK-3068-v0.patch
>
>
> I'd like to add some more logs related to the remove vs local file read
> operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Issue Comment Deleted] (OAK-3073) Make preftech of index files as default option
[
https://issues.apache.org/jira/browse/OAK-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-3073:
-
Comment: was deleted
(was: Applied the patch affecting {{LuceneIndexEditor}} in
http://svn.apache.org/r1689004)
> Make preftech of index files as default option
> ---
>
> Key: OAK-3073
> URL: https://issues.apache.org/jira/browse/OAK-3073
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.3.3
>
>
> As part of OAK-3069 a new feature is being provided to enable pre fetching of
> index files to local directory. To start with this would need to be
> explicitly. Once it is found to be stable we should enable by default.
> To start with we just enable it by default on trunk
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3073) Make preftech of index files as default option
[
https://issues.apache.org/jira/browse/OAK-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14613150#comment-14613150
]
Chetan Mehrotra commented on OAK-3073:
--
Applied the patch affecting {{LuceneIndexEditor}} in
http://svn.apache.org/r1689004
> Make preftech of index files as default option
> ---
>
> Key: OAK-3073
> URL: https://issues.apache.org/jira/browse/OAK-3073
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.3.3
>
>
> As part of OAK-3069 a new feature is being provided to enable pre fetching of
> index files to local directory. To start with this would need to be
> explicitly. Once it is found to be stable we should enable by default.
> To start with we just enable it by default on trunk
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3069) Provide option to eagerly copy the new index files in CopyOnRead
[
https://issues.apache.org/jira/browse/OAK-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14613149#comment-14613149
]
Chetan Mehrotra commented on OAK-3069:
--
Applied the patch in http://svn.apache.org/r1689003
On suggestion from [~mmarth] changed the config logic to be based on OSGi (and
globally applicable) instead of defining it on index. If later need is felt to
enable this on per index basis we can take care of that later on.
OSGi config exposed is a boolean property {{prefetchIndexFiles}} which is false
by default. With OAK-3073 we can decide if this needs to be enabled by default
or not
> Provide option to eagerly copy the new index files in CopyOnRead
>
>
> Key: OAK-3069
> URL: https://issues.apache.org/jira/browse/OAK-3069
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.2.3, 1.3.3, 1.0.17
>
> Attachments: OAK-3069+logs-v2.patch, OAK-3069+logs.patch,
> OAK-3069.patch
>
>
> Currently the CopyOnRead feature lazily starts copying the index files upon
> first access. This at times does not work efficiently because the index gets
> updated frequently and hence by the time copy is done index would have moved
> on. Due to this most of the index read turn out to be remote reads and cause
> slowness
> We should provide an option on per index basis to enable copying the files
> eagerly before exposing the new index to the query engine such that any read
> done is done locally
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3073) Make preftech of index files as default option
Chetan Mehrotra created OAK-3073: Summary: Make preftech of index files as default option Key: OAK-3073 URL: https://issues.apache.org/jira/browse/OAK-3073 Project: Jackrabbit Oak Issue Type: Improvement Components: lucene Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Priority: Minor Fix For: 1.3.3 As part of OAK-3069 a new feature is being provided to enable pre fetching of index files to local directory. To start with this would need to be explicitly. Once it is found to be stable we should enable by default. To start with we just enable it by default on trunk -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3069) Provide option to eagerly copy the new index files in CopyOnRead
[ https://issues.apache.org/jira/browse/OAK-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14613111#comment-14613111 ] Chetan Mehrotra commented on OAK-3069: -- Ideally this should be seen only for first time at time of start. As index is only reopened if something is written to it. Another case it can happen is when CopyOnWrite is enabled > Provide option to eagerly copy the new index files in CopyOnRead > > > Key: OAK-3069 > URL: https://issues.apache.org/jira/browse/OAK-3069 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.2.3, 1.3.3, 1.0.17 > > Attachments: OAK-3069+logs-v2.patch, OAK-3069+logs.patch, > OAK-3069.patch > > > Currently the CopyOnRead feature lazily starts copying the index files upon > first access. This at times does not work efficiently because the index gets > updated frequently and hence by the time copy is done index would have moved > on. Due to this most of the index read turn out to be remote reads and cause > slowness > We should provide an option on per index basis to enable copying the files > eagerly before exposing the new index to the query engine such that any read > done is done locally -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3072) LuceneIndexEditorTest#copyOnWriteAndLocks failing on windows
[
https://issues.apache.org/jira/browse/OAK-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612980#comment-14612980
]
Amit Jain commented on OAK-3072:
cc/ [~chetanm]
> LuceneIndexEditorTest#copyOnWriteAndLocks failing on windows
>
>
> Key: OAK-3072
> URL: https://issues.apache.org/jira/browse/OAK-3072
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.3.1
> Environment: windows
>Reporter: Amit Jain
>
> LuceneIndexEditorTest#copyOnWriteAndLocks regularly fails on windows with the
> following exception
> {noformat}
> copyOnWriteAndLocks(org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest)
> Time elapsed: 0.19 sec <<< ERROR!
> org.apache.jackrabbit.oak.api.CommitFailedException: OakLucene0003: Failed to
> index the node /test
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.addOrUpdate(LuceneIndexEditor.java:297)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.leave(LuceneIndexEditor.java:191)
> at
> org.apache.jackrabbit.oak.spi.commit.CompositeEditor.leave(CompositeEditor.java:74)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeAdded(EditorDiff.java:130)
> at
> org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:52)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:54)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest.copyOnWriteAndLocks(LuceneIndexEditorTest.java:376)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
> at org.junit.rules.RunRules.evaluate(RunRules.java:18)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
> Caused by: java.io.IOException: Cannot overwrite:
> C:\Users\amjain\AppData\Local\Temp\junit516545657673578972\2c26b46b68ffc68ff99b453c1
[jira] [Created] (OAK-3072) LuceneIndexEditorTest#copyOnWriteAndLocks failing on windows
Amit Jain created OAK-3072:
--
Summary: LuceneIndexEditorTest#copyOnWriteAndLocks failing on
windows
Key: OAK-3072
URL: https://issues.apache.org/jira/browse/OAK-3072
Project: Jackrabbit Oak
Issue Type: Bug
Components: lucene
Affects Versions: 1.3.1
Environment: windows
Reporter: Amit Jain
LuceneIndexEditorTest#copyOnWriteAndLocks regularly fails on windows with the
following exception
{noformat}
copyOnWriteAndLocks(org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest)
Time elapsed: 0.19 sec <<< ERROR!
org.apache.jackrabbit.oak.api.CommitFailedException: OakLucene0003: Failed to
index the node /test
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.addOrUpdate(LuceneIndexEditor.java:297)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.leave(LuceneIndexEditor.java:191)
at
org.apache.jackrabbit.oak.spi.commit.CompositeEditor.leave(CompositeEditor.java:74)
at
org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeAdded(EditorDiff.java:130)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:52)
at
org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:54)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest.copyOnWriteAndLocks(LuceneIndexEditorTest.java:376)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
at org.junit.rules.RunRules.evaluate(RunRules.java:18)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.io.IOException: Cannot overwrite:
C:\Users\amjain\AppData\Local\Temp\junit516545657673578972\2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae\1\_1.fdt
at
org.apache.lucene.store.FSDirectory.ensureCanWrite(FSDirectory.java:293)
at
org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:282)
at
org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnWriteDirectory$COWLocalFileReference.createOutput(IndexCopier.java:788)
at
org.apache.jackrabbit.oak.plu
[jira] [Commented] (OAK-3064) Oak Run Main.java passing values in the wrong order
[ https://issues.apache.org/jira/browse/OAK-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612952#comment-14612952 ] Michael Dürig commented on OAK-3064: What error you do get? Do you have a stack trace? Or even better a test case demonstrating the problem? > Oak Run Main.java passing values in the wrong order > --- > > Key: OAK-3064 > URL: https://issues.apache.org/jira/browse/OAK-3064 > Project: Jackrabbit Oak > Issue Type: Bug > Components: upgrade >Reporter: Jason E Bailey >Priority: Minor > > in org.apache.jackrabbit.oak.run.Main.java on line 952 the following method > is being called: > RepositoryContext.create(RepositoryConfig.create(dir, xml)); > The RepositoryConfig method signature is `create(File xml, File dir)` this is > creating an error when running the upgrade process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3064) Oak Run Main.java passing values in the wrong order
[ https://issues.apache.org/jira/browse/OAK-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-3064: --- Component/s: upgrade > Oak Run Main.java passing values in the wrong order > --- > > Key: OAK-3064 > URL: https://issues.apache.org/jira/browse/OAK-3064 > Project: Jackrabbit Oak > Issue Type: Bug > Components: upgrade >Reporter: Jason E Bailey >Priority: Minor > > in org.apache.jackrabbit.oak.run.Main.java on line 952 the following method > is being called: > RepositoryContext.create(RepositoryConfig.create(dir, xml)); > The RepositoryConfig method signature is `create(File xml, File dir)` this is > creating an error when running the upgrade process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
