Re: Best way to check Solr index for completeness

2010-09-29 Thread dshvadskiy
Actually retrieving 1000 docs via search isn't that bad. Turned out it takes under 1 sec. I still like the idea of using TermComponent and will use it in the future if number of docs in the index will grow. Thanks for all suggestions. Dmitriy -- View this message in context: http://lucene.47206

Re: Best way to check Solr index for completeness

2010-09-29 Thread dshvadskiy
Regenerating index is a slow operation due to limitation of the source systems. We run several complex SQL statements to generate 1 Solr document. Full reindex takes about 24 hours. -- View this message in context: http://lucene.472066.n3.nabble.com/Best-way-to-check-Solr-index-for-completenes

Re: Best way to check Solr index for completeness

2010-09-29 Thread dshvadskiy
Using TermComponent is an interesting suggestion. However my understanding it will work only for unique terms. For example compare database primary key with Solr id field. A variation of that is to calculate some kind of unique record hash and store it in the index.Then retrieve id and hash via T

Re: Best way to check Solr index for completeness

2010-09-28 Thread dshvadskiy
That will certainly work for most recent updates but I need to compare entire index. Dmitriy Luke Crouch wrote: > > Is there a 1:1 ratio of db records to solr documents? If so, couldn't you > simply select the most recent updated record from the db and check to make > sure the corresponding sol