Re: reg: efficient querying using solr

2013-06-12 Thread gururaj kosuru
imator-lu > cene-solr.xls > > > From: "gururaj kosuru" > Sent: Wednesday, June 12, 2013 12:28 AM > To: "solr-user" > Subject: Re: reg: efficient querying using solr > > Thanks Walter, Shawn and Otis for the

Re: reg: efficient querying using solr

2013-06-11 Thread Chris Morley
This might help (indirectly): http://svn.apache.org/repos/asf/lucene/dev/trunk/dev-tools/size-estimator-lu cene-solr.xls From: "gururaj kosuru" Sent: Wednesday, June 12, 2013 12:28 AM To: "solr-user" Subject: Re: reg: efficient

Re: reg: efficient querying using solr

2013-06-11 Thread gururaj kosuru
Thanks Walter, Shawn and Otis for the assistance, I will look into tuning the parameters by experimenting as seems to be the only way to go. On 11 June 2013 19:17, Shawn Heisey wrote: > On 6/11/2013 12:15 AM, gururaj kosuru wrote: > > How can one calculate an ideal max shard size for a solr cor

Re: reg: efficient querying using solr

2013-06-11 Thread Shawn Heisey
On 6/11/2013 12:15 AM, gururaj kosuru wrote: > How can one calculate an ideal max shard size for a solr core instance if I > am running a cloud with multiple systems of 4GB? That question is impossible to answer without experimentation, but here's a good starting point. That's all it is, a starti

Re: reg: efficient querying using solr

2013-06-11 Thread Otis Gospodnetic
You can't calculate it precisely, but you can test it. This is where EC2 is handy. Otis Solr & ElasticSearch Support http://sematext.com/ On Jun 11, 2013 2:16 AM, "gururaj kosuru" wrote: > How can one calculate an ideal max shard size for a solr core instance if I > am running a cloud with multi

Re: reg: efficient querying using solr

2013-06-10 Thread gururaj kosuru
How can one calculate an ideal max shard size for a solr core instance if I am running a cloud with multiple systems of 4GB? Thanks On 11 June 2013 11:18, Walter Underwood wrote: > An index does not need to fit into the heap. But a 4GB machine is almost > certainly too small to run Solr with 4

Re: reg: efficient querying using solr

2013-06-10 Thread Walter Underwood
An index does not need to fit into the heap. But a 4GB machine is almost certainly too small to run Solr with 40 million documents. wunder On Jun 10, 2013, at 10:36 PM, gururaj kosuru wrote: > Hi Walter, > thanks for replying. Do you mean that it is necessary for > the index to

Re: reg: efficient querying using solr

2013-06-10 Thread gururaj kosuru
Hi Walter, thanks for replying. Do you mean that it is necessary for the index to fit into the heap? if so, will a heap size that is greater than the actual RAM size slow down the queries? Thanks, Gururaj On 11 June 2013 10:36, Walter Underwood wrote: > 2GB is a rather small h

Re: reg: efficient querying using solr

2013-06-10 Thread Walter Underwood
2GB is a rather small heap. Our production systems run with 8GB and smaller indexes than that. Our dev and test systems run with 6GB heaps. wunder On Jun 10, 2013, at 9:52 PM, gururaj kosuru wrote: > Hello, > I have recently started using solr 3.4 and have a standalone > system deployed

reg: efficient querying using solr

2013-06-10 Thread gururaj kosuru
Hello, I have recently started using solr 3.4 and have a standalone system deployed that has 40,000,000 data rows with 3 indexed fields totalling around 9 GB. I have given a Heap size of 2GB and I run the instance on Tomcat on an i7 system with 4 GB RAM. My queries involve searching among