Gora Mohanty-3 wrote: > > On Wed, Sep 15, 2010 at 2:31 PM, yklxmas <yklx...@gmail.com> wrote: >> >> [...] >> Basically we have a file system that have many xml files to be indexed by >> solr. However, users might make changes to the files by using another >> editorial system that will export xml to the file system. After xml is >> exported, a call to solr to trigger full import. > >>>What do you mean by "full import" in this context? Are you simply POSTing >>>the XML file to Solr, or do you have a custom data import handler for which >>>you trigger a full-import? > > I'm using standard data import handler with file data source and xpath > processor. so my script will be calling > http://host:8983/solr/dataimport?command=full-import > >> >> I've got a few question: >> >> if full import is triggered while solr is still busy importing from the >> previous call, what will happen? > >>>If a full-import is still running, the standard DataImportHandler interface: >>>solr/dataimport?command=full-import will inform you that an import is still >>>running, and a fresh import will not be started. > > Does the fresh import go in a queue or simply won't start at all? If it > won't start, that means I need to find out the status and start again if > necessary. > >> shall i use delta-import instead of full-import as most of xml files may >> remain unchanged? > [...] > >>>How about having a script that runs periodically (say, at off-peak hours where >>>files are not being edited), checks the last-modified time of each file, and if >>>the file has been modified since the last import, POSTs the file to Solr? Seems >>>like this would be a simple solution. > > we need to push the changes out immediately. i need to rethink the process > and suggest my team a new approach more along what you suggested. > currently we haven't got that much data but constant full importing will > cause problems for sure. > > Regards, > Gora > > On Wed, Sep 15, 2010 at 2:31 PM, yklxmas <yklx...@gmail.com> wrote: > > [...] > Basically we have a file system that have many xml files to be indexed by > solr. However, users might make changes to the files by using another > editorial system that will export xml to the file system. After xml is > exported, a call to solr to trigger full import.
What do you mean by "full import" in this context? Are you simply POSTing the XML file to Solr, or do you have a custom data import handler for which you trigger a full-import? > > I've got a few question: > > if full import is triggered while solr is still busy importing from the > previous call, what will happen? If a full-import is still running, the standard DataImportHandler interface: solr/dataimport?command=full-import will inform you that an import is still running, and a fresh import will not be started. > shall i use delta-import instead of full-import as most of xml files may > remain unchanged? [...] How about having a script that runs periodically (say, at off-peak hours where files are not being edited), checks the last-modified time of each file, and if the file has been modified since the last import, POSTs the file to Solr? Seems like this would be a simple solution. Regards, Gora -- View this message in context: http://lucene.472066.n3.nabble.com/How-will-solr-behave-if-data-importing-is-called-while-another-importing-operation-is-still-ongoing-tp1478544p1479011.html Sent from the Solr - User mailing list archive at Nabble.com.
