Hi Mark,
I am still getting an OOM even after increasing the heap to 1024.
The docset I have is
numDocs : 1138976 maxDoc : 1180554
Not sure how much more I would need. Is there any other way out of this. I
noticed another interesting behavior. I have a Solr setup on a personal Box
where I try out a lot of different configuration and stuff before I even roll
the changes out to dev. This server has been running with a similar indexed
data for a lot longer than the dev box and it seems to have fetched the results
out properly.
This box is a windows 2 core processor with just about a gig of memory and the
whole 1024 megs have been allocated to heap. The dev is a linux with over 2
Gigs of memory and 1024 allocated to heap now. :S
-Sundar
> Date: Tue, 22 Jul 2008 13:17:40 -0700> From: [EMAIL PROTECTED]> To:
> solr-user@lucene.apache.org> Subject: Re: Out of memory on Solr sorting> >
> Mark,> > Question: how much memory I need for 25,000,000 docs if I do a sort
> by > <string> field, 256 bytes. 6.4Gb?> > > Quoting Mark Miller <[EMAIL
> PROTECTED]>:> > > Because to sort efficiently, Solr loads the term to sort on
> for each> > doc in the index into an array. For ints,longs, etc its just an
> array> > the size of the number of docs in your index (i believe deleted or>
> > not). For a String its an array to hold each unique string and an array> >
> of ints indexing into the String array.> >> > So if you do a sort, and search
> for something that only gets 1 doc as a> > hit...your still loading up that
> field cache for every single doc in> > your index on the first search. With
> solr, this happens in the> > background as it warms up the searcher. The end
> story is, you need more> > RAM to accommodate the sort most likely...have you
> upped your xmx> > setting? I think you can roughly say a 2 million doc index
> would need> > 40-50 MB (depending and rough, but to give an idea) per field
> your> > sorting on.> >> > - Mark> >> > >
_________________________________________________________________
Wish to Marry Now? Click Here to Register FREE
http://www.shaadi.com/registration/user/index.php?ptnr=mhottag
