Re: Planning Solr migration to production: clean and autoSoftCommit

2015-07-10 Thread Wenbin Wang
Hi Erick, Scheduling the indexing job is not an issue. The question is how to push the index to other two slave instances while the polling from other two slave instance needs to be manipulated. In the first option you proposed, I need to detect if the indexing job has completed, and force replic

Re: How to do a Data sharding for data in a database table

2015-06-25 Thread Wenbin Wang
. Regards, On Thu, Jun 25, 2015 at 1:27 PM, Shawn Heisey wrote: > On 6/25/2015 10:27 AM, Wenbin Wang wrote: > > To clarify the work: > > > > We are very early in the investigative phase, and the indexing is NOT > done > > continuously. > > > > I indexe

Re: How to do a Data sharding for data in a database table

2015-06-25 Thread Wenbin Wang
aken by various Solr > Solr components and may point to where the slowdown is coming from. > > Best, > Erick > > On Thu, Jun 25, 2015 at 9:48 AM, Wenbin Wang wrote: > > Hi Erick, > > > > The configuration is largely the default one, and I have not made much > &

Re: How to do a Data sharding for data in a database table

2015-06-25 Thread Wenbin Wang
Hi Erick, The configuration is largely the default one, and I have not made much change. I am also quite new to Solr although I have a lot of experience in other search products. The whole list of fields need to be retrieved, so I do not have much of a choice. The total size of the index files is

Re: How to do a Data sharding for data in a database table

2015-06-19 Thread Wenbin Wang
to sharding? > > > > To that end, what do you see when you set debugQuery=true? Where does > solr spend the time? My guess would be in the grouping and sorting steps, > but which? Sometime the schema details matter for performance. Folks on > this list can help with that.

Re: How to do a Data sharding for data in a database table

2015-06-19 Thread Wenbin Wang
I have enough RAM (30G) and Hard disk (1000G). It is not I/O bound or computer disk bound. In addition, the Solr was started with maximal 4G for JVM, and index size is < 2G. In a typical test, I made sure enough free RAM of 10G was available. I have not tuned any parameter in the configuration, it