Re: data-import runned by cron job withou wating the end of the previous one
On Mon, Sep 22, 2008 at 9:19 PM, sunnyfr <[EMAIL PROTECTED]> wrote: > > Hi, > There is something wierd : > I've plan cron job every 5mn which heat delta-import's url and works fine : > The point is : It does look like if it doesn't check every data for > updating > or creating a new one : > Because every 5mn the delta importa is started again : (even like if > delta-import is not done) > That should not be happening. Why do you feel it is starting again without waiting for the previous import to finish? > > idle > > − > > 0:2:23.885 > 1 > 1863146 > 0 > 0 > 2008-09-22 17:40:01 > 2008-09-22 17:40:01 > > I'm confused by this output. How frequently do you update your database? How many rows are modified in the database in that 5 minute period? What is the type of your last modified column in the database on which you use for identifying the deltas? > > and I wonder if it does come from my data-config file parameters : > which is adaptive : > >driver="com.mysql.jdbc.Driver" > url="jdbc:mysql://master.books.com/books" > user="solr" > password="tah1Axie" >batchSize="-1" > responseBuffering="adaptive"/> > > Thanks, > The part on responseBuffering is not applicable for MySQL so you can remove that. -- Regards, Shalin Shekhar Mangar.
Re: data-import runned by cron job withou wating the end of the previous one
when I try without adaptive parameter I've OOME: HTTP Status 500 - Java heap space java.lang.OutOfMemoryError: Java heap space Shalin Shekhar Mangar wrote: > > On Mon, Sep 22, 2008 at 9:19 PM, sunnyfr <[EMAIL PROTECTED]> wrote: > >> >> Hi, >> There is something wierd : >> I've plan cron job every 5mn which heat delta-import's url and works fine >> : >> The point is : It does look like if it doesn't check every data for >> updating >> or creating a new one : >> Because every 5mn the delta importa is started again : (even like if >> delta-import is not done) >> > > That should not be happening. Why do you feel it is starting again without > waiting for the previous import to finish? > > >> >> idle >> >> − >> >> 0:2:23.885 >> 1 >> 1863146 >> 0 >> 0 >> 2008-09-22 17:40:01 >> 2008-09-22 17:40:01 >> >> > > I'm confused by this output. How frequently do you update your database? > How > many rows are modified in the database in that 5 minute period? > > What is the type of your last modified column in the database on which you > use for identifying the deltas? > > >> >> and I wonder if it does come from my data-config file parameters : >> which is adaptive : >> >> > driver="com.mysql.jdbc.Driver" >> url="jdbc:mysql://master.books.com/books" >> user="solr" >> password="tah1Axie" >> batchSize="-1" >> responseBuffering="adaptive"/> >> >> Thanks, >> > > The part on responseBuffering is not applicable for MySQL so you can > remove > that. > > -- > Regards, > Shalin Shekhar Mangar. > > -- View this message in context: http://www.nabble.com/data-import-runned-by-cron-job-withou-wating-the-end-of-the-previous-one-tp19610823p19622383.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: data-import runned by cron job withou wating the end of the previous one
When I try without adaptive parameter I've an out of memory error. Shalin Shekhar Mangar wrote: > > On Mon, Sep 22, 2008 at 9:19 PM, sunnyfr <[EMAIL PROTECTED]> wrote: > >> >> Hi, >> There is something wierd : >> I've plan cron job every 5mn which heat delta-import's url and works fine >> : >> The point is : It does look like if it doesn't check every data for >> updating >> or creating a new one : >> Because every 5mn the delta importa is started again : (even like if >> delta-import is not done) >> > > That should not be happening. Why do you feel it is starting again without > waiting for the previous import to finish? > > >> >> idle >> >> − >> >> 0:2:23.885 >> 1 >> 1863146 >> 0 >> 0 >> 2008-09-22 17:40:01 >> 2008-09-22 17:40:01 >> >> > > I'm confused by this output. How frequently do you update your database? > How > many rows are modified in the database in that 5 minute period? > > What is the type of your last modified column in the database on which you > use for identifying the deltas? > > >> >> and I wonder if it does come from my data-config file parameters : >> which is adaptive : >> >> > driver="com.mysql.jdbc.Driver" >> url="jdbc:mysql://master.books.com/books" >> user="solr" >> password="tah1Axie" >> batchSize="-1" >> responseBuffering="adaptive"/> >> >> Thanks, >> > > The part on responseBuffering is not applicable for MySQL so you can > remove > that. > > -- > Regards, > Shalin Shekhar Mangar. > > -- View this message in context: http://www.nabble.com/data-import-runned-by-cron-job-withou-wating-the-end-of-the-previous-one-tp19610823p19622498.html Sent from the Solr - User mailing list archive at Nabble.com.
