Re: Solr 4.2.1 higher memory footprint vs Solr 3.5

Hi All, I work with Sandeep M, so continued to his comments. We did observe a memory growth. We use jdk1.6.0_45 with CMS. We see this issue because of large document size. With large i mean our single document has large multivalued fields. We found that JIRA LUCENE-4995

Re: Solr 4.2.1 higher memory footprint vs Solr 3.5

This is exactly what we did for a clients (alas using Elasticsearch). We then observed better performance through SPM. We used the latest Oracle JVM. Otis Solr & ElasticSearch Support http://sematext.com/ On Jun 7, 2013 2:55 AM, "Bernd Fehling" wrote: > Hi Shawn, > > I also had CMS with tons of

Re: Solr 4.2.1 higher memory footprint vs Solr 3.5

Hi Shawn, I also had CMS with tons of tuning options but still had once in a while bigger GC pause. After switching to JDK7 I tried G1GC with no other options and it runs perfekt. With CMS I saw that old and young generation where growing until they "had to do" a GC. This produces the sawtooth and

Re: Solr 4.2.1 higher memory footprint vs Solr 3.5

On 6/6/2013 3:50 AM, Bernd Fehling wrote: > What helped me a lot was switching to G1GC. > Faster, smoother, very little ripple, nearly no sawtooth. When I tried G1, it did indeed produce a better looking memory graph, but it didn't do anything about my GC pauses. They were several seconds with ju

Re: Solr 4.2.1 higher memory footprint vs Solr 3.5

Am 05.06.2013 18:09, schrieb SandeepM: > /So we see the jagged edge waveform which keeps climbing (GC cycles don't > completely collect memory over time). Our test has a short capture from > real traffic and we are replaying that via solrmeter./ > > Any idea why the memory climbs over time. Th

Re: Solr 4.2.1 higher memory footprint vs Solr 3.5

/So we see the jagged edge waveform which keeps climbing (GC cycles don't completely collect memory over time). Our test has a short capture from real traffic and we are replaying that via solrmeter./ Any idea why the memory climbs over time. The GC should cleanup after data is shipped back. Co

Re: Solr 4.2.1 higher memory footprint vs Solr 3.5

Shawn: You're right, I thought I'd seen it as a option but I think I was confusing really old solr. Thanks for catching, having gotten it wrong once I'm sure I'll remember it better for next time! Erick On Tue, Jun 4, 2013 at 1:57 PM, SandeepM wrote: > Thanks Eric and Shawn, > > Your explanat

Re: Solr 4.2.1 higher memory footprint vs Solr 3.5

Thanks Eric and Shawn, Your explanations help understand where SOLR may be spending its time. Sounds like compression can be a CPU and heap hog. (I'll try to confirm this with the heapdumps) Initially we tried to keep the JVM heap sizes the same on both Solr 3.5 and 4.2.1, which was around 3GB ,

Re: Solr 4.2.1 higher memory footprint vs Solr 3.5

On 6/4/2013 6:06 AM, Erick Erickson wrote: > About shipping data back to the client. Well, you > say the documents are very large Solr 4.2 > automatically compresses stored data, you might > be spending a lot if time decompressing the data. > You could try turning that option off. Other than Erick

Re: Solr 4.2.1 higher memory footprint vs Solr 3.5

While the schema may be the same, did you also closely examine solrconfig.xml, particularly cache settings? Memory _should_ have gone down significantly, especially if you're dealing with much textual data or string types. So I'm guessing something innocent in your configs is showing this differen

Solr 4.2.1 higher memory footprint vs Solr 3.5

Hi, Using the same schema for both Solr 3.5 and Solr 4.2.1 and posting the same data to both these server, and the memory requirements seem to have gone up sharply during request handling. . Requests come in at around 200QPS. . Document sizes are very large but that did not seem to be a problem w