Hi George,
I'm thinking that this was SOLR having an issue.
For some background, check out this thread:
http://www.mail-archive.com/dspace-tech@lists.sourceforge.net/msg11062.html
The two imbalances currently going on are writing to solr:
- too frequently: usage events, which end user traffic hitting your site.
possibly 10+ commits per second
- too infrequently: statistics importer. possibly too large at
potentially millions of records from log importing in a single commit.
solr.add(document) is a cheap operation, as is just puts something in queue
to be processed later.
solr.commit() is an expensive operation, which you want to call "just
right".
Since we don't really know whats right, I suggest we let solr do whats good
enough for it, and that would be autoCommit.
You can set that up with some
defaults: /dspace/solr/statistics/conf/solrconfig.xml
maxDocs - number of updates since last commit is greater than this
maxTime - oldest uncommited update (in ms) is this long ago
<autoCommit>
<maxDocs>10000</maxDocs>
<maxTime>1000</maxTime>
</autoCommit>
If you rely on autoCommit, we can then remove some of the
handholding/lockstepping of solr.add(doc), immediately followed by
solr.commit() which is terrible for solr performance (on busy instances).
So, check out activating autoCommit, and see if things improve. I would
suggest that solr autocommit becomes the default for future releases of
DSpace, so I'll need some agreement on this.
Peter Dietz
Systems Developer/Engineer
Ohio State University Libraries
On Mon, Sep 20, 2010 at 10:28 PM, George Stanley Kozak <g...@cornell.edu>wrote:
> Hi...
>
> I just migrated my production system to DSpace 1.6.2 from DSpace 1.5.2.
> Everything went well, until I did the stats-log-importer for the solr
> statistics.
> It ran for a long time and then I received this error:
>
> About to commit data to solr...Exception: Error opening new searcher.
> exceeded limit of maxWarmingSearchers=2. Try again later.
>
> Does anyone know what caused this error? I didn't see this when I upgraded
> my test system so I am assuming it has to do with the size of my production
> logs. IF so, what can I do to fix this?
>
>
>
> George Kozak
> Digital Library Specialist
> Division of Library Information Technologies (DLIT)
> 501 Olin Library
> Cornell University
> Ithaca, NY 14853
> 607-255-8924
>
> ------------------------------------------------------------------------------
> Start uncovering the many advantages of virtual appliances
> and start using them to simplify application deployment and
> accelerate your shift to cloud computing.
> http://p.sf.net/sfu/novell-sfdev2dev
> _______________________________________________
> DSpace-tech mailing list
> DSpace-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspace-tech
>
------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev
_______________________________________________
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
