Yonik,
I updated my solrconfig time based only as follows.
<autoCommit>
<maxTime>300000</maxTime>
</autoCommit>
<autoSoftCommit>
<maxTime>1000</maxTime>
</autoSoftCommit>
And changed my soft commit script to the first case.
while [ 1 ]; do
echo "Soft commit applied!"
wget -O /dev/null '
http://localhost:8080/solr-jak/dataimport?command=delta-import&commit=false'
2>/dev/null
sleep 1
done
After full-import, I inserted 420 new records in a minute. (7 new records
per second) And softCommitted every second as we can see in solrconfig.xml.
It seems that after all solr can return only 326 of these new 420 records.
Index files should not change every second, is it true? (After inserting
420 records if I call delta-import with commit true, all these records can
be seen in solr results)
Thanks,
Jak
On Thu, Nov 17, 2011 at 12:14 PM, Yonik Seeley
<yo...@lucidimagination.com>wrote:
> On Thu, Nov 17, 2011 at 11:48 AM, Jak Akdemir <jakde...@gmail.com> wrote:
> > 2) I am sure about delta-queries configured well. Full-Import is
> completed
> > in 40 secs for 400000 docs. And delta's are in 1 sec for 15 new records.
> > Also I checked it. There is no problem in it.
>
> That's 10,000 docs/sec. If you configure a soft commit for every 15
> documents, that means solr is trying to do 666 commits/sec.
> Autocommit by number of docs rarely makes sense anymore - I'd suggest
> configuring both soft and hard commits based on time only.
>
> -Yonik
> http://www.lucidimagination.com
>
