Re: How to reindex lucene

[Roy Teeuwen]

Hey Chetan,

Ok I’ll log a ticket with daycare. Thanks for the help :)!

Greets
Roy
> On 06 Jan 2016, at 15:25, chetan mehrotra [via Apache Sling] 
>  wrote:
> 
> Thanks Julian for jumping in! OAK-3504 is different. So far we have 
> not seen such corruption reported on Segment 
> 
> @Roy - Would be helpful if you can get a support case opened so that 
> we can get handle to repository content (if possible) which would 
> allow us to investigate better 
> Chetan Mehrotra 
> 
> 
> On Wed, Jan 6, 2016 at 5:48 PM, Julian Sedding <[hidden email] 
> > wrote:
> 
> > I missed the last two messages. The issue mentioned is fixed in 1.0.23 
> > (you are running 10.0.22). Since Chetan fixed it he may know much 
> > better than I do if you may be observing the same issue ;) 
> > 
> > Regards 
> > Julian 
> > 
> > On Wed, Jan 6, 2016 at 1:15 PM, Julian Sedding <[hidden email] 
> > > wrote: 
> >> Hi Roy 
> >> 
> >> The issue you observe may be 
> >> https://issues.apache.org/jira/browse/OAK-3504 
> >> . I don't know which Oak 
> >> version you are running, compare to the issue's fix version. If you 
> >> are running a "fixed" version, please report the issue on the Oak 
> >> list. Thanks. 
> >> 
> >> Regards 
> >> Julian 
> >> 
> >> 
> >> On Wed, Jan 6, 2016 at 12:58 PM, Roy Teeuwen <[hidden email] 
> >> > wrote: 
> >>> Hey Julian, 
> >>> 
> >>> I indeed already took this action, it just recreated the index folder but 
> >>> the error messages keep coming. 
> >>> 
> >>> The only way to “solve" it is by temporary disabling the oak-lucene 
> >>> bundle so that I can access the JCR Explorer with decent performance, set 
> >>> the reindex property for oak:index/lucene on true and then restart the 
> >>> oak-lucene bundle. During the recreation of the Lucene index you still 
> >>> see the exceptions being thrown in the error log but from the moment it 
> >>> finishes the reindexing the error messages disappear. 
> >>> 
> >>> Shall I put this on the mailing list for Jackrabbit oak? 
> >>> 
> >>> Thanks! 
> >>> Roy 
> >>> 
>  On 06 Jan 2016, at 12:47, Julian Sedding <[hidden email] 
>  > wrote: 
>  
>  Hi Roy 
>  
>  This relates more to Jackrabbit Oak then to Sling, so you might get 
>  better assistance there. 
>  
>  The Lucene-Index implementation in Oak copies the index data from the 
>  repository to the local file system as an I/O optimization. Therefore 
>  the files on the file system are redundant. 
>  
>  You can try the following: 
>  - shut down your Sling instance 
>  - delete /opt/aem_author/crx-quickstart/repository/index 
>  - start your Sling instance 
>  
>  If all goes as expected, you will see that the folder 
>  /opt/aem_author/crx-quickstart/repository/index is recreated and the 
>  log messages disappear. If this is not the case, please report back 
>  with your observations. 
>  
>  Hope this helps. 
>  
>  Regards 
>  Julian 
>  
>  
>  On Wed, Jan 6, 2016 at 10:14 AM, Roy Teeuwen <[hidden email] 
>  > wrote: 
> > I am having a problem with my lucene index, it is throwing the 
> > exception 
> > shown in the logs below. Last time I got this solved by setting the 
> > reindex 
> > property on true for the oak:index/lucene but now my system is so slow 
> > because it throws the exception so many times per second that I can't 
> > even 
> > get into the explorer to set the property on true. Is there a way to 
> > let my 
> > lucene index reindex by changing a property on the filesystem? Or is 
> > there a 
> > fix for this exception because reindexing every time also doesn't seem 
> > to be 
> > a correct way of handling this. 
> > 
> > 06.01.2016 09:55:16.310 *WARN* [oak-lucene-0] 
> > org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier 
> > [/oak:index/lucene] Error occurred while copying file [segments_1rt] 
> > from 
> > Directory for /oak:index/lucene to 
> > MMapDirectory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
> >  
> > lockFactory=NativeFSLockFactory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
> >  
> > java.io.FileNotFoundException: segments_1rt 
> >at 
> > org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory.openInput(OakDirectory.java:133)
> >  
> >at org.apache.lucene.store.Directory.copy(Directory.java:185) 
> >at 
> > org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.copyFilesToLocal(IndexCopier.java:420)
> >  
> >at 
> > org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.access$600(IndexCopier.java:296)
> >  
> >at 
> > org.apa

Re: How to reindex lucene

[Chetan Mehrotra]

Thanks Julian for jumping in! OAK-3504 is different. So far we have
not seen such corruption reported on Segment

@Roy - Would be helpful if you can get a support case opened so that
we can get handle to repository content (if possible) which would
allow us to investigate better
Chetan Mehrotra


On Wed, Jan 6, 2016 at 5:48 PM, Julian Sedding  wrote:
> I missed the last two messages. The issue mentioned is fixed in 1.0.23
> (you are running 10.0.22). Since Chetan fixed it he may know much
> better than I do if you may be observing the same issue ;)
>
> Regards
> Julian
>
> On Wed, Jan 6, 2016 at 1:15 PM, Julian Sedding  wrote:
>> Hi Roy
>>
>> The issue you observe may be
>> https://issues.apache.org/jira/browse/OAK-3504. I don't know which Oak
>> version you are running, compare to the issue's fix version. If you
>> are running a "fixed" version, please report the issue on the Oak
>> list. Thanks.
>>
>> Regards
>> Julian
>>
>>
>> On Wed, Jan 6, 2016 at 12:58 PM, Roy Teeuwen  wrote:
>>> Hey Julian,
>>>
>>> I indeed already took this action, it just recreated the index folder but 
>>> the error messages keep coming.
>>>
>>> The only way to “solve" it is by temporary disabling the oak-lucene bundle 
>>> so that I can access the JCR Explorer with decent performance, set the 
>>> reindex property for oak:index/lucene on true and then restart the 
>>> oak-lucene bundle. During the recreation of the Lucene index you still see 
>>> the exceptions being thrown in the error log but from the moment it 
>>> finishes the reindexing the error messages disappear.
>>>
>>> Shall I put this on the mailing list for Jackrabbit oak?
>>>
>>> Thanks!
>>> Roy
>>>
 On 06 Jan 2016, at 12:47, Julian Sedding  wrote:

 Hi Roy

 This relates more to Jackrabbit Oak then to Sling, so you might get
 better assistance there.

 The Lucene-Index implementation in Oak copies the index data from the
 repository to the local file system as an I/O optimization. Therefore
 the files on the file system are redundant.

 You can try the following:
 - shut down your Sling instance
 - delete /opt/aem_author/crx-quickstart/repository/index
 - start your Sling instance

 If all goes as expected, you will see that the folder
 /opt/aem_author/crx-quickstart/repository/index is recreated and the
 log messages disappear. If this is not the case, please report back
 with your observations.

 Hope this helps.

 Regards
 Julian


 On Wed, Jan 6, 2016 at 10:14 AM, Roy Teeuwen  wrote:
> I am having a problem with my lucene index, it is throwing the exception
> shown in the logs below. Last time I got this solved by setting the 
> reindex
> property on true for the oak:index/lucene but now my system is so slow
> because it throws the exception so many times per second that I can't even
> get into the explorer to set the property on true. Is there a way to let 
> my
> lucene index reindex by changing a property on the filesystem? Or is 
> there a
> fix for this exception because reindexing every time also doesn't seem to 
> be
> a correct way of handling this.
>
> 06.01.2016 09:55:16.310 *WARN* [oak-lucene-0]
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier
> [/oak:index/lucene] Error occurred while copying file [segments_1rt] from
> Directory for /oak:index/lucene to
> MMapDirectory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
> lockFactory=NativeFSLockFactory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
> java.io.FileNotFoundException: segments_1rt
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory.openInput(OakDirectory.java:133)
>at org.apache.lucene.store.Directory.copy(Directory.java:185)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.copyFilesToLocal(IndexCopier.java:420)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.access$600(IndexCopier.java:296)
>at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory$1.run(IndexCopier.java:379)
>at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>at java.lang.Thread.run(Thread.java:745)
> 06.01.2016 09:55:16.310 *WARN* [oak-lucene-0]
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier
> [/oak:index/lucene] Error occurred while copying file [segments_1ru] from
> Directory for /oak:index/lucene to
> MMapDirectory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ec

Re: How to reindex lucene

[Julian Sedding]

I missed the last two messages. The issue mentioned is fixed in 1.0.23
(you are running 10.0.22). Since Chetan fixed it he may know much
better than I do if you may be observing the same issue ;)

Regards
Julian

On Wed, Jan 6, 2016 at 1:15 PM, Julian Sedding  wrote:
> Hi Roy
>
> The issue you observe may be
> https://issues.apache.org/jira/browse/OAK-3504. I don't know which Oak
> version you are running, compare to the issue's fix version. If you
> are running a "fixed" version, please report the issue on the Oak
> list. Thanks.
>
> Regards
> Julian
>
>
> On Wed, Jan 6, 2016 at 12:58 PM, Roy Teeuwen  wrote:
>> Hey Julian,
>>
>> I indeed already took this action, it just recreated the index folder but 
>> the error messages keep coming.
>>
>> The only way to “solve" it is by temporary disabling the oak-lucene bundle 
>> so that I can access the JCR Explorer with decent performance, set the 
>> reindex property for oak:index/lucene on true and then restart the 
>> oak-lucene bundle. During the recreation of the Lucene index you still see 
>> the exceptions being thrown in the error log but from the moment it finishes 
>> the reindexing the error messages disappear.
>>
>> Shall I put this on the mailing list for Jackrabbit oak?
>>
>> Thanks!
>> Roy
>>
>>> On 06 Jan 2016, at 12:47, Julian Sedding  wrote:
>>>
>>> Hi Roy
>>>
>>> This relates more to Jackrabbit Oak then to Sling, so you might get
>>> better assistance there.
>>>
>>> The Lucene-Index implementation in Oak copies the index data from the
>>> repository to the local file system as an I/O optimization. Therefore
>>> the files on the file system are redundant.
>>>
>>> You can try the following:
>>> - shut down your Sling instance
>>> - delete /opt/aem_author/crx-quickstart/repository/index
>>> - start your Sling instance
>>>
>>> If all goes as expected, you will see that the folder
>>> /opt/aem_author/crx-quickstart/repository/index is recreated and the
>>> log messages disappear. If this is not the case, please report back
>>> with your observations.
>>>
>>> Hope this helps.
>>>
>>> Regards
>>> Julian
>>>
>>>
>>> On Wed, Jan 6, 2016 at 10:14 AM, Roy Teeuwen  wrote:
 I am having a problem with my lucene index, it is throwing the exception
 shown in the logs below. Last time I got this solved by setting the reindex
 property on true for the oak:index/lucene but now my system is so slow
 because it throws the exception so many times per second that I can't even
 get into the explorer to set the property on true. Is there a way to let my
 lucene index reindex by changing a property on the filesystem? Or is there 
 a
 fix for this exception because reindexing every time also doesn't seem to 
 be
 a correct way of handling this.

 06.01.2016 09:55:16.310 *WARN* [oak-lucene-0]
 org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier
 [/oak:index/lucene] Error occurred while copying file [segments_1rt] from
 Directory for /oak:index/lucene to
 MMapDirectory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
 lockFactory=NativeFSLockFactory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
 java.io.FileNotFoundException: segments_1rt
at
 org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory.openInput(OakDirectory.java:133)
at org.apache.lucene.store.Directory.copy(Directory.java:185)
at
 org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.copyFilesToLocal(IndexCopier.java:420)
at
 org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.access$600(IndexCopier.java:296)
at
 org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory$1.run(IndexCopier.java:379)
at
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
 06.01.2016 09:55:16.310 *WARN* [oak-lucene-0]
 org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier
 [/oak:index/lucene] Error occurred while copying file [segments_1ru] from
 Directory for /oak:index/lucene to
 MMapDirectory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
 lockFactory=NativeFSLockFactory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
 java.io.FileNotFoundException: segments_1ru
at
 org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory.openInput(OakDirectory.java:133)
at org.apache.lucene.store.Directory.copy(Directory.java:185)
at
 org.apache.jackrabbit.oak.plugins.index.luc

Re: How to reindex lucene

[Roy Teeuwen]

Ah cool, thanks for the help!

Greetings,
Roy
> On 06 Jan 2016, at 13:15, Julian Sedding-3 [via Apache Sling] 
>  wrote:
> 
> Hi Roy 
> 
> The issue you observe may be 
> https://issues.apache.org/jira/browse/OAK-3504 
> . I don't know which Oak 
> version you are running, compare to the issue's fix version. If you 
> are running a "fixed" version, please report the issue on the Oak 
> list. Thanks. 
> 
> Regards 
> Julian 
> 
> 
> On Wed, Jan 6, 2016 at 12:58 PM, Roy Teeuwen <[hidden email] 
> > wrote:
> 
> > Hey Julian, 
> > 
> > I indeed already took this action, it just recreated the index folder but 
> > the error messages keep coming. 
> > 
> > The only way to “solve" it is by temporary disabling the oak-lucene bundle 
> > so that I can access the JCR Explorer with decent performance, set the 
> > reindex property for oak:index/lucene on true and then restart the 
> > oak-lucene bundle. During the recreation of the Lucene index you still see 
> > the exceptions being thrown in the error log but from the moment it 
> > finishes the reindexing the error messages disappear. 
> > 
> > Shall I put this on the mailing list for Jackrabbit oak? 
> > 
> > Thanks! 
> > Roy 
> > 
> >> On 06 Jan 2016, at 12:47, Julian Sedding <[hidden email] 
> >> > wrote: 
> >> 
> >> Hi Roy 
> >> 
> >> This relates more to Jackrabbit Oak then to Sling, so you might get 
> >> better assistance there. 
> >> 
> >> The Lucene-Index implementation in Oak copies the index data from the 
> >> repository to the local file system as an I/O optimization. Therefore 
> >> the files on the file system are redundant. 
> >> 
> >> You can try the following: 
> >> - shut down your Sling instance 
> >> - delete /opt/aem_author/crx-quickstart/repository/index 
> >> - start your Sling instance 
> >> 
> >> If all goes as expected, you will see that the folder 
> >> /opt/aem_author/crx-quickstart/repository/index is recreated and the 
> >> log messages disappear. If this is not the case, please report back 
> >> with your observations. 
> >> 
> >> Hope this helps. 
> >> 
> >> Regards 
> >> Julian 
> >> 
> >> 
> >> On Wed, Jan 6, 2016 at 10:14 AM, Roy Teeuwen <[hidden email] 
> >> > wrote: 
> >>> I am having a problem with my lucene index, it is throwing the exception 
> >>> shown in the logs below. Last time I got this solved by setting the 
> >>> reindex 
> >>> property on true for the oak:index/lucene but now my system is so slow 
> >>> because it throws the exception so many times per second that I can't 
> >>> even 
> >>> get into the explorer to set the property on true. Is there a way to let 
> >>> my 
> >>> lucene index reindex by changing a property on the filesystem? Or is 
> >>> there a 
> >>> fix for this exception because reindexing every time also doesn't seem to 
> >>> be 
> >>> a correct way of handling this. 
> >>> 
> >>> 06.01.2016 09:55:16.310 *WARN* [oak-lucene-0] 
> >>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier 
> >>> [/oak:index/lucene] Error occurred while copying file [segments_1rt] from 
> >>> Directory for /oak:index/lucene to 
> >>> MMapDirectory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
> >>>  
> >>> lockFactory=NativeFSLockFactory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
> >>>  
> >>> java.io.FileNotFoundException: segments_1rt 
> >>>at 
> >>> org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory.openInput(OakDirectory.java:133)
> >>>  
> >>>at org.apache.lucene.store.Directory.copy(Directory.java:185) 
> >>>at 
> >>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.copyFilesToLocal(IndexCopier.java:420)
> >>>  
> >>>at 
> >>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.access$600(IndexCopier.java:296)
> >>>  
> >>>at 
> >>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory$1.run(IndexCopier.java:379)
> >>>  
> >>>at 
> >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>>  
> >>>at 
> >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> >>>  
> >>>at java.lang.Thread.run(Thread.java:745) 
> >>> 06.01.2016 09:55:16.310 *WARN* [oak-lucene-0] 
> >>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier 
> >>> [/oak:index/lucene] Error occurred while copying file [segments_1ru] from 
> >>> Directory for /oak:index/lucene to 
> >>> MMapDirectory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
> >>>  
> >>> lockFactory=NativeFSLockFactory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
> >>>  
> >>> java.io.FileNotFoundException: segments_1ru 
> >>> 

Re: How to reindex lucene

[Julian Sedding]

Hi Roy

The issue you observe may be
https://issues.apache.org/jira/browse/OAK-3504. I don't know which Oak
version you are running, compare to the issue's fix version. If you
are running a "fixed" version, please report the issue on the Oak
list. Thanks.

Regards
Julian


On Wed, Jan 6, 2016 at 12:58 PM, Roy Teeuwen  wrote:
> Hey Julian,
>
> I indeed already took this action, it just recreated the index folder but the 
> error messages keep coming.
>
> The only way to “solve" it is by temporary disabling the oak-lucene bundle so 
> that I can access the JCR Explorer with decent performance, set the reindex 
> property for oak:index/lucene on true and then restart the oak-lucene bundle. 
> During the recreation of the Lucene index you still see the exceptions being 
> thrown in the error log but from the moment it finishes the reindexing the 
> error messages disappear.
>
> Shall I put this on the mailing list for Jackrabbit oak?
>
> Thanks!
> Roy
>
>> On 06 Jan 2016, at 12:47, Julian Sedding  wrote:
>>
>> Hi Roy
>>
>> This relates more to Jackrabbit Oak then to Sling, so you might get
>> better assistance there.
>>
>> The Lucene-Index implementation in Oak copies the index data from the
>> repository to the local file system as an I/O optimization. Therefore
>> the files on the file system are redundant.
>>
>> You can try the following:
>> - shut down your Sling instance
>> - delete /opt/aem_author/crx-quickstart/repository/index
>> - start your Sling instance
>>
>> If all goes as expected, you will see that the folder
>> /opt/aem_author/crx-quickstart/repository/index is recreated and the
>> log messages disappear. If this is not the case, please report back
>> with your observations.
>>
>> Hope this helps.
>>
>> Regards
>> Julian
>>
>>
>> On Wed, Jan 6, 2016 at 10:14 AM, Roy Teeuwen  wrote:
>>> I am having a problem with my lucene index, it is throwing the exception
>>> shown in the logs below. Last time I got this solved by setting the reindex
>>> property on true for the oak:index/lucene but now my system is so slow
>>> because it throws the exception so many times per second that I can't even
>>> get into the explorer to set the property on true. Is there a way to let my
>>> lucene index reindex by changing a property on the filesystem? Or is there a
>>> fix for this exception because reindexing every time also doesn't seem to be
>>> a correct way of handling this.
>>>
>>> 06.01.2016 09:55:16.310 *WARN* [oak-lucene-0]
>>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier
>>> [/oak:index/lucene] Error occurred while copying file [segments_1rt] from
>>> Directory for /oak:index/lucene to
>>> MMapDirectory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
>>> lockFactory=NativeFSLockFactory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
>>> java.io.FileNotFoundException: segments_1rt
>>>at
>>> org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory.openInput(OakDirectory.java:133)
>>>at org.apache.lucene.store.Directory.copy(Directory.java:185)
>>>at
>>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.copyFilesToLocal(IndexCopier.java:420)
>>>at
>>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.access$600(IndexCopier.java:296)
>>>at
>>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory$1.run(IndexCopier.java:379)
>>>at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>at java.lang.Thread.run(Thread.java:745)
>>> 06.01.2016 09:55:16.310 *WARN* [oak-lucene-0]
>>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier
>>> [/oak:index/lucene] Error occurred while copying file [segments_1ru] from
>>> Directory for /oak:index/lucene to
>>> MMapDirectory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
>>> lockFactory=NativeFSLockFactory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
>>> java.io.FileNotFoundException: segments_1ru
>>>at
>>> org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory.openInput(OakDirectory.java:133)
>>>at org.apache.lucene.store.Directory.copy(Directory.java:185)
>>>at
>>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.copyFilesToLocal(IndexCopier.java:420)
>>>at
>>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.access$600(IndexCopier.java:296)
>>>at
>>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory$1.run(IndexCopier.java:379)
>>>at
>>> java.util.concurrent.ThreadPoolExecutor.r

Re: How to reindex lucene

[Roy Teeuwen]

Hey Chetan,

It’s a SegmentMK and Oak is 1.0.22 (AEM 6.0 SP3).

Greets
Roy

> On 06 Jan 2016, at 13:09, chetan mehrotra [via Apache Sling] 
>  wrote:
> 
> On Wed, Jan 6, 2016 at 5:28 PM, Roy Teeuwen <[hidden email] 
> > wrote: 
> > Shall I put this on the mailing list for Jackrabbit oak? 
> 
> That would be better. Given the issue is being seen on AEM instance it 
> might also be better to go for AEM Support so that we can get more 
> data easily. btw is it on SegmentMK or Mongo? And whats the Oak 
> version in use here? 
> 
> Chetan Mehrotra 
> 
> 
> If you reply to this email, your message will be added to the discussion 
> below:
> http://apache-sling.73963.n3.nabble.com/How-to-reindex-lucene-tp4058168p4058185.html
>  
> 
> To unsubscribe from How to reindex lucene, click here 
> .
> NAML 
> 




--
View this message in context: 
http://apache-sling.73963.n3.nabble.com/How-to-reindex-lucene-tp4058168p4058186.html
Sent from the Sling - Users mailing list archive at Nabble.com.

Re: How to reindex lucene

[Chetan Mehrotra]

On Wed, Jan 6, 2016 at 5:28 PM, Roy Teeuwen  wrote:
> Shall I put this on the mailing list for Jackrabbit oak?

That would be better. Given the issue is being seen on AEM instance it
might also be better to go for AEM Support so that we can get more
data easily. btw is it on SegmentMK or Mongo? And whats the Oak
version in use here?

Chetan Mehrotra

Re: How to reindex lucene

[Roy Teeuwen]

Hey Julian,

I indeed already took this action, it just recreated the index folder but the 
error messages keep coming. 

The only way to “solve" it is by temporary disabling the oak-lucene bundle so 
that I can access the JCR Explorer with decent performance, set the reindex 
property for oak:index/lucene on true and then restart the oak-lucene bundle. 
During the recreation of the Lucene index you still see the exceptions being 
thrown in the error log but from the moment it finishes the reindexing the 
error messages disappear.

Shall I put this on the mailing list for Jackrabbit oak?

Thanks!
Roy

> On 06 Jan 2016, at 12:47, Julian Sedding  wrote:
> 
> Hi Roy
> 
> This relates more to Jackrabbit Oak then to Sling, so you might get
> better assistance there.
> 
> The Lucene-Index implementation in Oak copies the index data from the
> repository to the local file system as an I/O optimization. Therefore
> the files on the file system are redundant.
> 
> You can try the following:
> - shut down your Sling instance
> - delete /opt/aem_author/crx-quickstart/repository/index
> - start your Sling instance
> 
> If all goes as expected, you will see that the folder
> /opt/aem_author/crx-quickstart/repository/index is recreated and the
> log messages disappear. If this is not the case, please report back
> with your observations.
> 
> Hope this helps.
> 
> Regards
> Julian
> 
> 
> On Wed, Jan 6, 2016 at 10:14 AM, Roy Teeuwen  wrote:
>> I am having a problem with my lucene index, it is throwing the exception
>> shown in the logs below. Last time I got this solved by setting the reindex
>> property on true for the oak:index/lucene but now my system is so slow
>> because it throws the exception so many times per second that I can't even
>> get into the explorer to set the property on true. Is there a way to let my
>> lucene index reindex by changing a property on the filesystem? Or is there a
>> fix for this exception because reindexing every time also doesn't seem to be
>> a correct way of handling this.
>> 
>> 06.01.2016 09:55:16.310 *WARN* [oak-lucene-0]
>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier
>> [/oak:index/lucene] Error occurred while copying file [segments_1rt] from
>> Directory for /oak:index/lucene to
>> MMapDirectory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
>> lockFactory=NativeFSLockFactory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
>> java.io.FileNotFoundException: segments_1rt
>>at
>> org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory.openInput(OakDirectory.java:133)
>>at org.apache.lucene.store.Directory.copy(Directory.java:185)
>>at
>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.copyFilesToLocal(IndexCopier.java:420)
>>at
>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.access$600(IndexCopier.java:296)
>>at
>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory$1.run(IndexCopier.java:379)
>>at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>at java.lang.Thread.run(Thread.java:745)
>> 06.01.2016 09:55:16.310 *WARN* [oak-lucene-0]
>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier
>> [/oak:index/lucene] Error occurred while copying file [segments_1ru] from
>> Directory for /oak:index/lucene to
>> MMapDirectory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
>> lockFactory=NativeFSLockFactory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
>> java.io.FileNotFoundException: segments_1ru
>>at
>> org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory.openInput(OakDirectory.java:133)
>>at org.apache.lucene.store.Directory.copy(Directory.java:185)
>>at
>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.copyFilesToLocal(IndexCopier.java:420)
>>at
>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.access$600(IndexCopier.java:296)
>>at
>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory$1.run(IndexCopier.java:379)
>>at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>at java.lang.Thread.run(Thread.java:745)
>> 06.01.2016 09:55:16.572 *WARN* [qtp483471336-38]
>> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier
>> [/oak:index/lucene] Error occurred while copying file [_1sa.cfs] from
>> Directory for /oak:index/lucene to
>> MMapDir

Re: How to reindex lucene

[Julian Sedding]

Hi Roy

This relates more to Jackrabbit Oak then to Sling, so you might get
better assistance there.

The Lucene-Index implementation in Oak copies the index data from the
repository to the local file system as an I/O optimization. Therefore
the files on the file system are redundant.

You can try the following:
- shut down your Sling instance
- delete /opt/aem_author/crx-quickstart/repository/index
- start your Sling instance

If all goes as expected, you will see that the folder
/opt/aem_author/crx-quickstart/repository/index is recreated and the
log messages disappear. If this is not the case, please report back
with your observations.

Hope this helps.

Regards
Julian


On Wed, Jan 6, 2016 at 10:14 AM, Roy Teeuwen  wrote:
> I am having a problem with my lucene index, it is throwing the exception
> shown in the logs below. Last time I got this solved by setting the reindex
> property on true for the oak:index/lucene but now my system is so slow
> because it throws the exception so many times per second that I can't even
> get into the explorer to set the property on true. Is there a way to let my
> lucene index reindex by changing a property on the filesystem? Or is there a
> fix for this exception because reindexing every time also doesn't seem to be
> a correct way of handling this.
>
> 06.01.2016 09:55:16.310 *WARN* [oak-lucene-0]
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier
> [/oak:index/lucene] Error occurred while copying file [segments_1rt] from
> Directory for /oak:index/lucene to
> MMapDirectory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
> lockFactory=NativeFSLockFactory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
> java.io.FileNotFoundException: segments_1rt
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory.openInput(OakDirectory.java:133)
> at org.apache.lucene.store.Directory.copy(Directory.java:185)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.copyFilesToLocal(IndexCopier.java:420)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.access$600(IndexCopier.java:296)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory$1.run(IndexCopier.java:379)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 06.01.2016 09:55:16.310 *WARN* [oak-lucene-0]
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier
> [/oak:index/lucene] Error occurred while copying file [segments_1ru] from
> Directory for /oak:index/lucene to
> MMapDirectory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
> lockFactory=NativeFSLockFactory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
> java.io.FileNotFoundException: segments_1ru
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory.openInput(OakDirectory.java:133)
> at org.apache.lucene.store.Directory.copy(Directory.java:185)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.copyFilesToLocal(IndexCopier.java:420)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory.access$600(IndexCopier.java:296)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnReadDirectory$1.run(IndexCopier.java:379)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 06.01.2016 09:55:16.572 *WARN* [qtp483471336-38]
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier
> [/oak:index/lucene] Error occurred while copying file [_1sa.cfs] from
> Directory for /oak:index/lucene to
> MMapDirectory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
> lockFactory=NativeFSLockFactory@/opt/aem_author/crx-quickstart/repository/index/e5a943cdec3000bd8ce54924fd2070ab5d1d35b9ecf530963a3583d43bf28293/2
> java.io.EOFException: reached end of stream after reading 0 bytes; 717339
> bytes expected
> at com.google.common.io.ByteStreams.readFully(ByteStreams.java:697)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexFile.loadBlob(OakDirectory.java:261)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexFile.readBytes(OakDirectory.java:307)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.OakDirector