Re: Can we point a Solr server to index directory dynamically at runtime..

2009-09-25 Thread Michael
Are you storing (in addition to indexing) your data? Perhaps you could turn off storage on data older than 7 days (requires reindexing), thus losing the ability to return snippets but cutting down on your storage space and server count. I've experienced 10x decrease in space requirements and a

Re: Can we point a Solr server to index directory dynamically at runtime..

2009-09-25 Thread Silent Surfer
Hi Michael, We are storing all our data in addition to index, as we need to display those values to the user. So unfortunately we cannot go with the option stored=false, which could have potentially solved our issue. Appreciate any other pointers/suggestions Thanks, sS --- On Fri, 9/25/09,

Re: Can we point a Solr server to index directory dynamically at runtime..

2009-09-24 Thread Michael
Using a multicore approach, you could send a create a core named 'core3weeksold' pointing to '/datadirs/3weeksold' command to a live Solr, which would spin it up on the fly. Then you query it, and maybe keep it spun up until it's not queried for 60 seconds or something, then send a remove core

Re: Can we point a Solr server to index directory dynamically at runtime..

2009-09-24 Thread Chris Hostetter
: Using a multicore approach, you could send a create a core named : 'core3weeksold' pointing to '/datadirs/3weeksold' command to a live Solr, : which would spin it up on the fly. Then you query it, and maybe keep it : spun up until it's not queried for 60 seconds or something, then send a :

Re: Can we point a Solr server to index directory dynamically at runtime..

2009-09-24 Thread Silent Surfer
Hi, Thank you Michael and Chris for the response. Today after the mail from Michael, we tested with the dynamic loading of cores and it worked well. So we need to go with the hybrid approach of Multicore and Distributed searching. As per our testing, we found that a Solr instance with 20 GB

Can we point a Solr server to index directory dynamically at runtime..

2009-09-23 Thread Silent Surfer
Hi, Is there any way to dynamically point the Solr servers to an index/data directories at run time? We are generating 200 GB worth of index per day and we want to retain the index for approximately 1 month. So our idea is to keep the first 1 week of index available at anytime for the users