Re: Solr Merge during off peak times

2012-05-05 Thread Shawn Heisey
On 5/4/2012 8:10 PM, Lance Norskog wrote: Optimize takes a 'maxSegments' option. This tells it to stop when there are N segments instead of just one. If you use a very high mergeFactor and then call optimize with a sane number like 50, it only merges the little teeny segments. When I

Re: Solr Merge during off peak times

2012-05-04 Thread Lance Norskog
Optimize takes a 'maxSegments' option. This tells it to stop when there are N segments instead of just one. If you use a very high mergeFactor and then call optimize with a sane number like 50, it only merges the little teeny segments. On Thu, May 3, 2012 at 8:28 PM, Shawn Heisey

RE: Solr Merge during off peak times

2012-05-03 Thread Prakashganesh, Prabhu
Great, thanks Otis and Erick for your responses I will take a look at SPM Thanks Prabhu -Original Message- From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] Sent: 03 May 2012 00:02 To: solr-user@lucene.apache.org Subject: Re: Solr Merge during off peak times Hello Prabhu

Re: Solr Merge during off peak times

2012-05-03 Thread Erick Erickson
May 2012 13:15 To: solr-user@lucene.apache.org Subject: Re: Solr Merge during off peak times But again, with a master/slave setup merging should be relatively benign. And at 200M docs, having a M/S setup is probably indicated. Here's a good writeup of mergepolicy http

Re: Solr Merge during off peak times

2012-05-03 Thread Shawn Heisey
On 5/2/2012 5:54 AM, Prakashganesh, Prabhu wrote: We have a fairly large scale system - about 200 million docs and fairly high indexing activity - about 300k docs per day with peak ingestion rates of about 20 docs per sec. I want to work out what a good mergeFactor setting would be by testing

RE: Solr Merge during off peak times

2012-05-02 Thread Prakashganesh, Prabhu
@lucene.apache.org solr-user@lucene.apache.org Sent: Tuesday, May 1, 2012 8:45 AM Subject: Solr Merge during off peak times Hi,   I would like to know if there is a way to configure index merge policy in solr so that the merging happens during off peak hours. Can you please let me know if such a merge

Re: Solr Merge during off peak times

2012-05-02 Thread Erick Erickson
. Do you know if this can be done or if there is some tool for this? Thanks Prabhu -Original Message- From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] Sent: 01 May 2012 15:12 To: solr-user@lucene.apache.org Subject: Re: Solr Merge during off peak times Hi Prabhu, I don't

RE: Solr Merge during off peak times

2012-05-02 Thread Prakashganesh, Prabhu
the response times are etc.. Thanks Prabhu -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: 02 May 2012 12:40 To: solr-user@lucene.apache.org Subject: Re: Solr Merge during off peak times Why do you care? Merging is generally a background process, or are you doing

Re: Solr Merge during off peak times

2012-05-02 Thread Erick Erickson
-Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: 02 May 2012 12:40 To: solr-user@lucene.apache.org Subject: Re: Solr Merge during off peak times Why do you care? Merging is generally a background process, or are you doing heavy indexing? In a master/slave setup

RE: Solr Merge during off peak times

2012-05-02 Thread Prakashganesh, Prabhu
Subject: Re: Solr Merge during off peak times But again, with a master/slave setup merging should be relatively benign. And at 200M docs, having a M/S setup is probably indicated. Here's a good writeup of mergepolicy http://juanggrande.wordpress.com/2011/02/07/merge-policy-internals/ If you're

Re: Solr Merge during off peak times

2012-05-02 Thread Erick Erickson
is more static, perhaps? Thoughts? Thanks Prabhu -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: 02 May 2012 13:15 To: solr-user@lucene.apache.org Subject: Re: Solr Merge during off peak times But again, with a master/slave setup merging should

Re: Solr Merge during off peak times

2012-05-02 Thread Otis Gospodnetic
: Prakashganesh, Prabhu prabhu.prakashgan...@dowjones.com To: solr-user@lucene.apache.org solr-user@lucene.apache.org; Otis Gospodnetic otis_gospodne...@yahoo.com Sent: Wednesday, May 2, 2012 7:22 AM Subject: RE: Solr Merge during off peak times Ok, thanks Otis Another question on merging What

Re: Solr Merge during off peak times

2012-05-01 Thread Otis Gospodnetic
solr-user@lucene.apache.org Sent: Tuesday, May 1, 2012 8:45 AM Subject: Solr Merge during off peak times Hi,   I would like to know if there is a way to configure index merge policy in solr so that the merging happens during off peak hours. Can you please let me know if such a merge policy