Re: Recrawling with Solr backend

2011-07-15 Thread Chris Alexander
Hi Lewis, Sorry for the delay in responding - that clears those questions up thanks. For now we are working on a script to hopefully minimise the impact of the writes to the Solr index. We are also baking in deletions through the use of a Solr query and splitting separate domains out into their

Re: Recrawling with Solr backend

2011-07-14 Thread Chris Alexander
Hi Lewis, First of all, thanks for the fantastic reply, most useful. I am working on testing out the functions you mention, of which I was not previously aware. There are a few offshoot questions from this that the answers to which aren't immediately apparent. When a solrindex is run doing an

Re: Recrawling with Solr backend

2011-07-14 Thread lewis john mcgibbney
Pleas seem comments below On Thu, Jul 14, 2011 at 12:52 PM, Chris Alexander chris.alexan...@kusiri.com wrote: Hi Lewis, First of all, thanks for the fantastic reply, most useful. I am working on testing out the functions you mention, of which I was not previously aware. Yes there has been

Recrawling with Solr backend

2011-07-13 Thread Chris Alexander
Hi, I have been looking up re-crawling mechanisms with Nutch, and just about all I have come across is designed for pre-1.3 versions using the non-Solr index. We're using 1.3 with the Solr index (just because that was the latest version we downloaded to try out and we are already using Solr), and