On 11/09/2007, at 8:46 AM, Ryan McKinley wrote:
lucene opens a lot of files. It can easily get beyond 1024. (I
think the default). I'm no expert on how the file handling works,
but I think more files are open if you are searching and writing at
the same time.
If you can't increase the limit you can try:
<useCompoundFile>true</useCompoundFile>
It is slower, but if you are unable to change the ulimit on the
deployed machines....
I've done a bit of poking on the server and ulimit doesn't seem to be
the problem:
e2wiki:~$ ulimit
unlimited
e2wiki:~$ cat /proc/sys/fs/file-max
170355
So there's either something going on behind my back (quite possible,
it's a VM) or lucene is opening a really insane number of files. I
did check that those values were the same for the tomcat55 user that
Tomcat actually runs as. An lsof -p on the Tomcat process always
shows 40 files in use, the total open files sits around 1000-1500
even when reindexing all the content. I'll watch it a bit more over
time and see what happens.
I notice that Confluence recommends at least 200000 for the max file
limit, at least before they switched to compound indexing so it's
possible that the 170355 limit could be reached, but it seems
unlikely with our load.
If you need to use this in production soon, I'd suggest sticking
with 1.2 for a while. There has been a LOT of action in trunk and
it may be good to let it settle before upgrading a production system.
You should not need to upgrade to fix the write.lock and Too Many
Open Files problem. Try increasing ulimit or using a compoundfile
before upgrading.
We're quite a way off of real production, it's just internal use at
the moment (on the real product server, but we're a small company so
we can handle having some problems). I'll try out the current nightly
build and see how it goes, as much as anything out of interest but
probably won't pull new builds very often.
Thanks again,
Adrian Sutton
http://www.symphonious.net
