Hi,
We are sharing a Lucene index in a Linux cluster over an NFS share. We have
multiple servers reading and writing to the index.
I am getting regular lock exceptions e.g.
Lock obtain timed out:
NativeFSLock@/mnt/nfstest/repository/lucene/lock/lucene-2d3d31fa7f19eabb73d692df44087d81-n-write.lo
hi Patrick,
Mike is the expert in this, but until he gets in, can you add details on
the update pattern - note that the DeletionPolicy you describe below is not
(afaik) related to the write lock time-out issues you are facing. The
DeletionPolicy manages better the interaction between an IndexWrite
Hi Doron
Thanks for your reply.
I am working on the details of the update pattern. It will take me
some time as I cannot reproduce the issue on demand.
To answer your other questions, yes, we do have multiple writers. One
writer per node in the cluster.
I will post the results of my investig
Hi
As requested, I have been trying to improve the logging in the
application so I can give you more details of the update pattern.
I am using the Lucene Index Accessor contribution to co-ordinate the
readers and writers:
http://www.nabble.com/Fwd%3A-Contribution%3A-LuceneIndexAccessor-t17416.ht
Note that some Solr users have reported a similar issue.
https://issues.apache.org/jira/browse/SOLR-240
-Yonik
On 6/29/07, Patrick Kimber <[EMAIL PROTECTED]> wrote:
Hi
As requested, I have been trying to improve the logging in the
application so I can give you more details of the update patter
This is an interesting choice. Perhaps you have modified
LuceneIndexAccessor, but it seems to me (without knowing much about your
setup) that you would have odd reader behavior. On a 3 node system, if you
add docs with node 1 and 2 but not 3 and your doing searches against all 3
nodes, node 3 will
Hi Mark
Yes, thank you. I can see your point and I think we might have to pay
some attention to this issue.
But, we sometimes see this error on an NFS share within 2 minutes of
starting the test so I don't think this is the only problem.
Once again, thanks for the idea. I will certainly be lo
If your getting java.io.FileNotFoundException:
/mnt/nfstest/repository/lucene/lucene-icm-test-1-0/segments_h75 within 2
minutes, this is very odd indeed. That would seem to imply your deletion
policy is not working.
You might try just using one of the nodes as the writer. In Michaels
comments
Hi Mark
I just ran my test again... and the error occurred after 10 minutes -
which is the time when my deletion policy is triggered. So... I think
you might have found the answer to my problem.
I will spend more time looking at it on Monday.
Thank you very much for your help and enjoy your we
: We are sharing a Lucene index in a Linux cluster over an NFS share. We have
: multiple servers reading and writing to the index.
:
: I am getting regular lock exceptions e.g.
: Lock obtain timed out:
:
NativeFSLock@/mnt/nfstest/repository/lucene/lock/lucene-2d3d31fa7f19eabb73d692df44087d81-n-w
: Perhaps i'm missing something, but i thought NativeFSLock was not suitable
: for NFS? ... or is is this what "lockd" provides? (my NFS knowledge is
: very out of date)
Do'h!
I just read the docs for NativeFSLockFactory and noticed the "For example,
for NFS servers there sometimes must be a sepa
Patrick Kimber wrote:
> As requested, I have been trying to improve the
> logging in the application so I can give you more
> details of the update pattern.
>
> I am using the Lucene Index Accessor contribution
> to co-ordinate the readers and writers:
> http://www.nabble.com/Fwd%3A-Contribution%3
Yonik wrote:
> Note that some Solr users have reported a similar issue.
> https://issues.apache.org/jira/browse/SOLR-240
Seems the scenario there is without using native locks? -
"i get the stacktrace below ... with useNativeLocks turned off"
Mark Miller wrote:
> You might try just using one of the nodes as
> the writer. In Michaels comments, he always seems
> to mention the pattern of one writer many
> readers on nfs. In this case you could use
> no LockFactory and perhaps gain a little speed there.
One thing I would worry about if m
Never used the IndexAccessor patch, so I may be
wrong in the following.
No, let's fix it... /;->
Don't mean to wade in over my head here, but just to help out those that
have not used LuceneIndexAccessor.
I am fairly certain that using the LuceneIndexAccessor could easily
create the File
On 6/29/07, Doron Cohen <[EMAIL PROTECTED]> wrote:
> Note that some Solr users have reported a similar issue.
> https://issues.apache.org/jira/browse/SOLR-240
Seems the scenario there is without using native locks? -
"i get the stacktrace below ... with useNativeLocks turned off"
Yes... but th
Patrick Kimber wrote:
> I have been checking the application log. Just before the time when
> the lock file errors occur I found this log entry:
> [11:28:59] [ERROR] IndexAccessProvider
> java.io.FileNotFoundException:
> /mnt/nfstest/repository/lucene/lucene-icm-test-1-0/segments_h75 (No
> such
Hi
I have added more logging to my test application. I have two servers
writing to a shared Lucene index on an NFS partition...
Here is the logging from one server...
[10:49:18] [DEBUG] LuceneIndexAccessor closing cached writer
[10:49:18] [DEBUG] ExpirationTimeDeletionPolicy onCommit() delete
t
Re: Lucene 2.2, NFS, Lock obtain timed out
Hi
I have added more logging to my test application. I have two servers
writing to a shared Lucene index on an NFS partition...
Here is the logging from one server...
[10:49:18] [DEBUG] LuceneIndexAccessor closing cached writer
[
er" <[EMAIL PROTECTED]>
07/03/2007 03:47 PM
Please respond to
java-user@lucene.apache.org
To
java-user@lucene.apache.org
cc
Subject
Re: Lucene 2.2, NFS, Lock obtain timed out
Hi
I have added more logging to my test application. I have two servers
writing to a shared Lucene ind
to read the
indexes, first server has created.
Cheers,
Neeraj
"Patrick Kimber" <[EMAIL PROTECTED]>
07/03/2007 04:16 PM
Please respond to
java-user@lucene.apache.org
To
java-user@lucene.apache.org
cc
Subject
Re: Lucene 2.2, NFS, Lock obtain timed out
Hi
I am using th
"Patrick Kimber" <[EMAIL PROTECTED]> wrote:
> I am using the NativeFSLockFactory. I was hoping this would have
> stopped these errors.
I believe this is not a locking issue and NativeFSLockFactory should
be working correctly over NFS.
> Here is the whole of the stack trace:
>
> Caused by: java
Hi Michael
I am really pleased we have a potential fix. I will look out for the patch.
Thanks for your help.
Patrick
On 03/07/07, Michael McCandless <[EMAIL PROTECTED]> wrote:
"Patrick Kimber" <[EMAIL PROTECTED]> wrote:
> I am using the NativeFSLockFactory. I was hoping this would have
>
OK I opened issue LUCENE-948, and attached a patch & new 2.2.0 JAR.
Please make sure you use the "take2" versions (they have added
instrumentation to help us debug):
https://issues.apache.org/jira/browse/LUCENE-948
Patrick, could you please test the above "take2" JAR? Could you also call
Ind
Hi Michael
I am setting up the test with the "take2" jar and will let you know
the results as soon as I have them.
Thanks for your help
Patrick
On 03/07/07, Michael McCandless <[EMAIL PROTECTED]> wrote:
OK I opened issue LUCENE-948, and attached a patch & new 2.2.0 JAR.
Please make sure you u
Hi Michael
I have been running the test for over an hour without any problem.
The index writer log file is getting rather large so I cannot leave
the test running overnight. I will run the test again tomorrow
morning and let you know how it goes.
Thanks again...
Patrick
On 03/07/07, Patrick K
"Patrick Kimber" <[EMAIL PROTECTED]> wrote:
> I have been running the test for over an hour without any problem.
> The index writer log file is getting rather large so I cannot leave
> the test running overnight. I will run the test again tomorrow
> morning and let you know how it goes.
Ahhh, th
Hi Michael
Yes, there are many lines in the logs saying:
hit FileNotFoundException when loading commit "segment_X"; skipping
this commit point
...so it looks like the new code is working perfectly.
I am sorry to be vague... but how do I check which segments file is
opened when a new writer is cr
"Patrick Kimber" <[EMAIL PROTECTED]> wrote:
> Yes, there are many lines in the logs saying:
> hit FileNotFoundException when loading commit "segment_X"; skipping
> this commit point
> ...so it looks like the new code is working perfectly.
Super!
> I am sorry to be vague... but how do I check wh
Hi Michael
Just to let you know, I am on holiday for one week so will not be able
to send a progress report until I return.
I have deployed the new code to a test site so I will be informed if
the users notice any issues.
Thanks for your help
Patrick
On 04/07/07, Michael McCandless <[EMAIL P
Hi
We are still getting various issues on our Lucene indexes running on an NFS
share. It has taken me some time to find some useful information to report
to the mailing list.
I have created a test application which is running on two Linux servers.
The Lucene index is on an NFS share. After ru
>
> "pkimber" <[EMAIL PROTECTED]> wrote:
>
> > We are still getting various issues on our Lucene indexes running on
> > an NFS share. It has taken me some time to find some useful
> > information to report to the mailing list.
>
> Bummer!
>
> Can you zip up your test application that shows the iss
"pkimber" <[EMAIL PROTECTED]> wrote:
> We are still getting various issues on our Lucene indexes running on
> an NFS share. It has taken me some time to find some useful
> information to report to the mailing list.
Bummer!
Can you zip up your test application that shows the issue, as well as
t
"Patrick Kimber" <[EMAIL PROTECTED]> wrote:
> I cannot send you the source code without speaking to my manager
> first. I guess he would want me to change the code before sending it
> to you. You could have the log files now, but I expect you want to
> wait until the test application is ready t
34 matches
Mail list logo
