Hi Yonik,
Be that as it may, I'm seeing a steady increase in file handles used by that
process over an extended period of time (now 20+ minutes):
r...@duck6:~# lsof -p 22379 | wc
786 7714 108339
r...@duck6:~# lsof -p 22379 | wc
787 7723 108469
r...@duck6:~# lsof -p 22379 | wc
787 7723 108469
r...@duck6:~# lsof -p 22379 | wc
812 7948 111719
r...@duck6:~# lsof -p 22379 | wc
816 7984 112239
r...@duck6:~# lsof -p 22379 | wc
817 7993 112369
r...@duck6:~# lsof -p 22379 | wc
822 8038 113019
r...@duck6:~# lsof -p 22379 | wc
847 8308 116719
r...@duck6:~# lsof -p 22379 | wc
852 8353 117369
r...@duck6:~# lsof -p 22379 | wc
897 8803 123669
r...@duck6:~# lsof -p 22379 | wc
1022 10018 140819
r...@duck6:~#
This doesn't smell like spiky resource usage to me. It smells like a leak. ;-)
Karl
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of ext Yonik Seeley
Sent: Friday, September 10, 2010 2:08 PM
To: [email protected]
Subject: Re: Trunk file handle leak?
On Fri, Sep 10, 2010 at 1:51 PM, <[email protected]> wrote:
> (1) There are periodic commits, every 10,000 records.
> (2) I have no searcher/reader open at the same time, that I am aware of.
> This is a straight indexing task. (You ought to know, you wrote some of the
> code!)
A commit currently opens a new searcher in Solr.
It's not too hard to go past 1024 descriptors - either raise the limit
to 10240 or something, use the compound file format, or lower the
merge factor.
> (3) I *do* see auto warming being called, but it seems not to happen at the
> same time as a commit, but rather afterwards.
>
> Once it starts happening, this happens repeatedly on every commit.
This would also be expected - it's at a point where there are too many
files for your descriptors.
-Yonik
http://lucenerevolution.org Lucene/Solr Conference, Boston Oct 7-8
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
