On 10/9/07, David Whalen <[EMAIL PROTECTED]> wrote:
> I run a faceted query against a very large index on a
> regular schedule. Every now and then the query throws
> an out of heap space error, and we're sunk.
>
> So, naturally we increased the heap size and things worked
> well for a while and th
ed set for it
Dave
> -Original Message-
> From: Yonik Seeley [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, October 09, 2007 12:47 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Facets and running out of Heap Space
>
> On 10/9/07, David Whalen <[EMAIL PROTEC
On 10/9/07, David Whalen <[EMAIL PROTECTED]> wrote:
> > This is only used during the term enumeration method of
> > faceting (facet.field type faceting on multi-valued or
> > full-text fields).
>
> What if I'm faceting on just a plain String field? It's
> not full-text, and I don't have multiValue
: So, naturally we increased the heap size and things worked
: well for a while and then the errors would happen again.
: We've increased the initial heap size to 2.5GB and it's
: still happening.
is this the same 25,000,000 document index you mentioned before?
2.5GB of heap doesn't seem like mu
ltiValued?
_
david whalen
senior applications developer
eNR Services, Inc.
[EMAIL PROTECTED]
203-849-7240
> -Original Message-
> From: Yonik Seeley [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, October 09, 2007 2:14 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Facets
e_id to almost 100,000 for journalist_id.
Thanks very much for your help so far, Chris!
Dave
> -Original Message-
> From: Chris Hostetter [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, October 09, 2007 2:48 PM
> To: solr-user
> Subject: Re: Facets and running out of Heap S
the results are valuable.
In what way? I'm still not clear on what this does for me
> -Original Message-
> From: Ryan McKinley [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, October 09, 2007 4:01 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Facets a
what does the LukeReqeust Handler tell you about the # of
distinct terms in each field that you facet on?
Where would I find that?
check:
http://wiki.apache.org/solr/LukeRequestHandler
Make sure you have:
class="org.apache.solr.handler.admin.LukeRequestHandler" />
defined in solrconfig.x
David Whalen wrote:
Make sure you have:
class="org.apache.solr.handler.admin.LukeRequestHandler" />
defined in solrconfig.xml
What's the consequence of me changing the solrconfig.xml file?
Doesn't that cause a restart of solr?
editing solrconfig.xml does *not* restart solr.
But you need to
On 9-Oct-07, at 12:36 PM, David Whalen wrote:
stored="true" />
multiValued="true" />
stored="true" />
stored="true" />
I'm sure we could stop storing many of these columns, especially
if someone told me that would make a big difference.
I don't think that it would make a difference
d
documentCache).
Thanks,
Stu
-Original Message-
From: Mike Klaas <[EMAIL PROTECTED]>
Sent: Tuesday, October 9, 2007 9:30pm
To: solr-user@lucene.apache.org
Subject: Re: Facets and running out of Heap Space
On 9-Oct-07, at 12:36 PM, David Whalen wrote:
>(snip)
> I'm sure we could sto
On 9-Oct-07, at 7:53 PM, Stu Hood wrote:
Using the filter cache method on the things like media type and
location; this will occupy ~2.3MB of memory _per unique value_
Mike, how did you calculate that value? I'm trying to tune my
caches, and any equations that could be used to determine some
g the problem to another step in the
process?
DW
> -Original Message-
> From: Stu Hood [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, October 09, 2007 10:53 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Facets and running out of Heap Space
>
> > Using t
g minDf to a very high value
should always outperform such an approach.
-Mike
DW
-Original Message-
From: Stu Hood [mailto:[EMAIL PROTECTED]
Sent: Tuesday, October 09, 2007 10:53 PM
To: solr-user@lucene.apache.org
Subject: Re: Facets and running out of Heap Space
Using the fi
al Message-
> From: Mike Klaas [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, October 10, 2007 4:56 PM
> To: solr-user@lucene.apache.org
> Cc: stuhood
> Subject: Re: Facets and running out of Heap Space
>
> On 10-Oct-07, at 12:19 PM, David Whalen wrote:
>
> > It lo
Mike Klaas [mailto:[EMAIL PROTECTED]
Sent: Wednesday, October 10, 2007 4:56 PM
To: solr-user@lucene.apache.org
Cc: stuhood
Subject: Re: Facets and running out of Heap Space
On 10-Oct-07, at 12:19 PM, David Whalen wrote:
It looks now like I can't use facets the way I was hoping
to becaus
Wednesday, October 10, 2007 6:20 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Facets and running out of Heap Space
>
> On 10-Oct-07, at 2:40 PM, David Whalen wrote:
>
> > Accoriding to Yonik I can't use minDf because I'm faceting
> on a string
> >
On 10-Oct-07, at 3:46 PM, David Whalen wrote:
I'll see what I can do about that.
Truthfully, the most important facet we need is the one on
media_type, which has only 4 unique values. The second
most important one to us is location, which has about 30
unique values.
So, it would seem like we
On 10/10/07, Mike Klaas <[EMAIL PROTECTED]> wrote:
> Have you tried setting multivalued=true without reindexing? I'm not
> sure, but I think it will work.
Yes, that will work fine.
One thing that will change is the response format for stored fields
val1
instead of
val1
Hopefully in the future we
19 matches
Mail list logo
