Hmmm. It is suspicious that your index files change every second. If you change our cron task to update every 10 seconds, do the index files change every 10 seconds?
Regarding your question about "After a server restart last query results reserved. (In NRT they would disappear, right?)" not necessarily. If your <autoCommit> interval is exceeded, the soft commits will be committed to disk so your Solr restart would pick them up after restart. But if somehow you're getting a hard commit to happen every second, you should also be seeing a lot of segment merging going on, are you? I think I'd stop the cron job and execute this manually for a while in order to see exactly where the problem is. I'd go ahead and comment out the <autoCommit> section as well. That should give you a much more reproducible test scenario. Say you do that, issue your delta-import and immediately kill your server. When it starts up if you then see the delta-data, we should understand why. Because it sure would seem like the commit=false isn't doing what you expect. Erick On Thu, Nov 17, 2011 at 12:41 PM, Jak Akdemir <jakde...@gmail.com> wrote: > Yonik, > > I updated my solrconfig time based only as follows. > <autoCommit> > <maxTime>300000</maxTime> > </autoCommit> > > <autoSoftCommit> > <maxTime>1000</maxTime> > </autoSoftCommit> > > And changed my soft commit script to the first case. > while [ 1 ]; do > echo "Soft commit applied!" > wget -O /dev/null ' > http://localhost:8080/solr-jak/dataimport?command=delta-import&commit=false' > 2>/dev/null > sleep 1 > done > > After full-import, I inserted 420 new records in a minute. (7 new records > per second) And softCommitted every second as we can see in solrconfig.xml. > It seems that after all solr can return only 326 of these new 420 records. > Index files should not change every second, is it true? (After inserting > 420 records if I call delta-import with commit true, all these records can > be seen in solr results) > > Thanks, > > Jak > > On Thu, Nov 17, 2011 at 12:14 PM, Yonik Seeley > <yo...@lucidimagination.com>wrote: > >> On Thu, Nov 17, 2011 at 11:48 AM, Jak Akdemir <jakde...@gmail.com> wrote: >> > 2) I am sure about delta-queries configured well. Full-Import is >> completed >> > in 40 secs for 400000 docs. And delta's are in 1 sec for 15 new records. >> > Also I checked it. There is no problem in it. >> >> That's 10,000 docs/sec. If you configure a soft commit for every 15 >> documents, that means solr is trying to do 666 commits/sec. >> Autocommit by number of docs rarely makes sense anymore - I'd suggest >> configuring both soft and hard commits based on time only. >> >> -Yonik >> http://www.lucidimagination.com >> >
