Re: Segments and Memory Correlate?

2011-03-21 Thread danomano
yes we are having physical memory issues, but do does anyone know for a fact if there is direct correlation between be segment counts and RAM - memory? i.e. when the system begins a search on a segment, does it load that segments full index. Or does the system load All index data for all

Solr 1.4.1 missing CSV queryResponseWriter?

2011-03-21 Thread danomano
Hi folks, I was running 1.4.0, but today I'm trying to switch to 1.4.1 (which appears to work with Zoie), however when I try to retrieve data in CSV format, wt=csv does not appear to work anymore. Looking at the solrconfig.xml I see: (I tried pulling out the CSV entry, but that failed (class not

Segments and Memory Correlate?

2011-03-17 Thread danomano
Hi folks, I ran into problem today where I am no longer able to execute any queries :( due to Out of Memory issues. I am in the process of investigating the use of different mergeFactors, or even different merge policies all together. My question is if I have many segments (i.e. smaller sized

Re: Solr Hanging all of sudden with update/csv

2011-03-11 Thread danomano
Sweet, those links very very useful :). and should most definitely help :) One overriding concern I have: 1) if I were to simply update the config to use a different mergeFactor, and restart the solr server, (would it then adjust the segments accordingly?) or would I need to start from

Re: Solr Hanging all of sudden with update/csv

2011-03-09 Thread danomano
After About 4-5 hours the merge completed (ran out of heap)..as you suggested..it was having memory issues.. Read queries during the merge were working just fine (they were taking longer then normal ~30-60seconds). I think I need to do more reading on understanding the merge/optimization

Solr Hanging all of sudden with update/csv

2011-03-08 Thread danomano
Hi folks, I've been using solr for about 3 months. Our Solr install is a single node, and we have been injecting logging data into the solr server every couple of minutes, which each updating taking few minutes. Everything working fine until this morning, at which point it appeared that all

Re: Solr Hanging all of sudden with update/csv

2011-03-08 Thread danomano
Actually this is definitely not a ram issue. I have visualVM connected and MAX Ram available for the JavaVM is ~7GB, but the system is only using ~5.5GB, with a MAX so far of 6.5GB consumed. I think..well I'm guessing the system hit a merge threshold, but I can't tell for sure..I have seen the

Re: does solr support posting gzipped content?

2010-11-16 Thread danomano
sorry, yes by inject I simply mean post, I was hoping to get away without writing any 'native' solr code to upload gzip files, but its sounds like that is not possible. (The file's that I'm uploading (aka posting are CSV formatted). I will poke around and which solution

does solr support posting gzipped content?

2010-10-19 Thread danomano
Hi folks, I was wondering if there is any native support for posting gzipped files to solr? i.e. I'm testing a project where we inject our log files into solr for indexing, these logs files are gzipped, and I figure it would take less network bandwith to inject gzipped files directl. is there a