Yes my database is remote, mysql 5 and i'm using connector/J 5.1.7. My index
has 20000 documents. When i try to do lets say 14 updates it takes about 18
sec total. Here's the resulting log of the operation :
2009-03-25 15:53:57 org.apache.solr.handler.dataimport.JdbcDataSource$1 call
INFO: Time taken for getConnection(): 411
2009-03-25 15:53:59 org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Completed ModifiedRowKey for Entity: profil rows obtained : 14
2009-03-25 15:53:59 org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Completed DeletedRowKey for Entity: profil rows obtained : 0
2009-03-25 15:53:59 org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Completed parentDeltaQuery for Entity: profil
2009-03-25 15:54:00 org.apache.solr.core.SolrDeletionPolicy onInit
INFO: SolrDeletionPolicy.onInit: commits:num=1
commit{dir=/home/solr-tomcat/solr/data/index,segFN=segments_sb,version=1237322897338,generation=1019,filenames=[_uj.frq,
_uj.fdx, _uj.tii, _uj.nrm, _uj.tis, _uj.fnm, _uj.prx, segments_sb, _uj.fdt]
2009-03-25 15:54:00 org.apache.solr.core.SolrDeletionPolicy updateCommits
INFO: last commit = 1237322897338
2009-03-25 15:54:13 org.apache.solr.handler.dataimport.DocBuilder doDelta
INFO: Delta Import completed successfully BOTTLE NECK
2009-03-25 15:54:13 org.apache.solr.handler.dataimport.DocBuilder commit
INFO: Full Import completed successfully
2009-03-25 15:54:13 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start commit(optimize=true,waitFlush=false,waitSearcher=true)
2009-03-25 15:54:15 org.apache.solr.core.SolrDeletionPolicy onCommit
INFO: SolrDeletionPolicy.onCommit: commits:num=2
commit{dir=/home/solr-tomcat/solr/data/index,segFN=segments_sb,version=1237322897338,generation=1019,filenames=[_uj.frq,
_uj.fdx, _uj.tii, _uj.nrm, _uj.tis, _uj.fnm, _uj.prx, segments_sb, _uj.fdt]
commit{dir=/home/solr-tomcat/solr/data/index,segFN=segments_sc,version=1237322897339,generation=1020,filenames=[_ul.prx,
_ul.fnm, _ul.tii, _ul.fdt, _ul.nrm, _ul.fdx, _ul.tis, _ul.frq, segments_sc]
2009-03-25 15:54:15 org.apache.solr.core.SolrDeletionPolicy updateCommits
INFO: last commit = 1237322897339
2009-03-25 15:54:15 org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening searc...@3da850 main
When i do a full-import it is much faster. Take about 1 min to index 20000
documents. I tried to play a bit with the config but nothing seems to work
for the moment.
What i want to do is pretty interactive, my production db has 1,2M documents
and must be able to delta-import around 2k update every 5min. Is it
possible with the dataimporthandle to reach those kinda of number ?
Shalin Shekhar Mangar wrote:
>
> On Wed, Mar 25, 2009 at 2:25 AM, AlexxelA
> <alexandre.boudrea...@canoe.ca>wrote:
>
>>
>> Ok i'm ok with the fact the solr gonna do X request to database for X
>> update.. but when i try to run the delta-import command with 20000 row to
>> update is it normal that its kinda really slow ~ 1 document fetched / sec
>> ?
>>
>>
> Not really, I've seen 1000x faster. Try firing a few of those queries on
> the
> database directly. Are they slow? Is the database remote?
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
>
--
View this message in context:
http://www.nabble.com/Delta-import-tp22663196p22710222.html
Sent from the Solr - User mailing list archive at Nabble.com.
