Hi Avenka, *DataImportHandler* 1.) there is no configuration to add the last uniqueKeyField-Values to dataimport.properties 2.) you can use LogUpdateProcessor to log all "schema.printableUniqueKey(doc)" to log.info( ""+toLog + " 0 " + (elapsed) ) 3.) you can write your own LogUpdateProcessor to log only the last UniqueKey 4.) you can change DocBuilder#execute to store the uniqueKey in dataimport.properties
*max(id)* With TermsComponent you can easily ask for the first term in a field (so you could add a field with "10000000 - id" to find the last term in id). With solr 4.0 some index-codes will support "give me the last term" in a field: Fields#getUniqueTermCount() together with TermsEnum#seekExact(long) With solr 3.6 you can use TermsComponent together wir guessing a "terms.lower" to find the last term in a field. This should outran a "*:*" search with function max(id). Beste regards Karsten View this message in context: http://lucene.472066.n3.nabble.com/DataImport-using-last-indexed-id-or-getting-max-id-quickly-tp3993763.html -------- Original-Nachricht -------- > Datum: Sun, 8 Jul 2012 10:25:55 -0700 (PDT) > Von: avenka <ave...@gmail.com> > An: solr-user@lucene.apache.org > Betreff: DataImport using last_indexed_id or getting max(id) quickly > My understanding is that the DIH in solr only enters last_indexed_time in > dataimport.properties, but not say last_indexed_id for a primary key 'id'. > How can I efficiently get the max(id) (note that 'id' is an auto-increment > field in the database) ? Maintaining max(id) outside of solr is brittle > and > calling max(id) before each dataimport can take several minutes when the > index has several hundred million records. > > How can I either import based on ID or get max(id) quickly? I can not use > timestamp-based import because I get out-of-memory errors if/when solr > falls > behind and the suggested fixes online did not work for me. > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/DataImport-using-last-indexed-id-or-getting-max-id-quickly-tp3993763.html > Sent from the Solr - User mailing list archive at Nabble.com.
