Re: DIH dataimport.properties with

2010-04-20 Thread Shawn Heisey
On 4/20/2010 9:09 PM, caman wrote: Shawn, Is this your custom implementation? "For a delta-import, minDid comes from the maxDid value stored after the last successful import." Are you updating the dataTable after the import was successful? How did you handle this? I have similar scenario and y

Re: DIH dataimport.properties with

2010-04-20 Thread Michael Tibben
Hey Shawn, Thanks for the tip. This will work nicely. I totally missed the ability to use request parameters using ${dataimporter.request.* } This way, I can maintain my own last_index_time timestamp outside of the DIH properties file. Much appreciation Regards, Michael On 21/04/10 09

RE: DIH dataimport.properties with

2010-04-20 Thread caman
will work for my use-case as well thanks From: Shawn Heisey-4 [via Lucene] [mailto:ml-node+738653-1765413222-124...@n3.nabble.com] Sent: Tuesday, April 20, 2010 4:35 PM To: caman Subject: Re: DIH dataimport.properties with Michael, The SolrEntityProcessor looks very

Re: DIH dataimport.properties with

2010-04-20 Thread Lance Norskog
Please file a JIRA asking for the ability to direct what is saved in the DIH properties file. It should be possible to use your own timestamp or generated id. On 4/20/10, Shawn Heisey wrote: > Michael, > > The SolrEntityProcessor looks very intriguing, but it won't work with > the released 1.4 ve

Re: DIH dataimport.properties with

2010-04-20 Thread Shawn Heisey
Michael, The SolrEntityProcessor looks very intriguing, but it won't work with the released 1.4 version. If that's OK with you and it looks like it'll do what you want, feel free to ignore the rest of this. I'm also using MySQL as an import source for Solr. I was unable to use the last_ind

Re: DIH dataimport.properties with

2010-04-18 Thread Michael Tibben
: Thu, April 15, 2010 10:15:14 PM Subject: DIH dataimport.properties with Hi, I am using the DIH to import data from a mysql slave. However, the slave sometimes runs behind the master. The delay is variable, most of the time it is in sync, but sometimes can run behind by a few mi

Re: DIH dataimport.properties with

2010-04-18 Thread Michael Tibben
http://sematext.com/ :: Solr - Lucene - Nutch Hadoop ecosystem search :: http://search-hadoop.com/ - Original Message From: Michael Tibben To: solr-user@lucene.apache.org Sent: Thu, April 15, 2010 10:15:14 PM Subject: DIH dataimport.properties with Hi, I am using the DIH to i

Re: DIH dataimport.properties with

2010-04-17 Thread Lance Norskog
r@lucene.apache.org >> Sent: Thu, April 15, 2010 10:15:14 PM >> Subject: DIH dataimport.properties with >> >> Hi, > > I am using the DIH to import data from a mysql slave. However, the >> slave sometimes runs behind the master. The delay is variable, most

Re: DIH dataimport.properties with

2010-04-16 Thread Otis Gospodnetic
:15:14 PM > Subject: DIH dataimport.properties with > > Hi, I am using the DIH to import data from a mysql slave. However, the > slave sometimes runs behind the master. The delay is variable, most of the > time > it is in sync, but sometimes can run behind by a few minutes. Th

DIH dataimport.properties with

2010-04-15 Thread Michael Tibben
Hi, I am using the DIH to import data from a mysql slave. However, the slave sometimes runs behind the master. The delay is variable, most of the time it is in sync, but sometimes can run behind by a few minutes. This is a problem, because DIH uses dataimport.properties to determine the last