Michael McCandless [mailto:luc...@mikemccandless.com]
> Sent: Friday, May 14, 2010 11:23 AM
> To: java-user@lucene.apache.org
> Subject: Re: IndexWriter and memory usage
>
> The patch looks correct.
>
> The 16 MB RAM buffer means the sum of the shared char[], byte[] and
> Post
memory during our large
document indexing runs.
Thanks for your help Michael in resolving this.
-Original Message-
From: Michael McCandless [mailto:luc...@mikemccandless.com]
Sent: Friday, May 14, 2010 11:23 AM
To: java-user@lucene.apache.org
Subject: Re: IndexWriter and memory usage
The
ook at this data
> quicker than the screen shots I send.
>
> -Original Message-
> From: Michael McCandless [mailto:luc...@mikemccandless.com]
> Sent: Monday, May 10, 2010 2:27 AM
> To: java-user@lucene.apache.org
> Subject: Re: IndexWriter and memory usage
>
> H...
H...
Your usage (searching for old doc & updating it, to add new fields) is fine.
But: what memory usage do you see if you open a searcher, and search
for all docs, but don't open an IndexWriter? We need to tease apart
the IndexReader vs IndexWriter memory usage you are seeing. Also, can
yo
Sent: Thursday, April 29, 2010 2:07 PM
To: java-user@lucene.apache.org
Subject: Re: IndexWriter and memory usage
OK I think you may be hitting this:
https://issues.apache.org/jira/browse/LUCENE-2422
Since you have very large docs, the reuse that's done by
IndexInput/Output is tying u
ing this issue.
>
> -Original Message-
> From: Michael McCandless [mailto:luc...@mikemccandless.com]
> Sent: Tuesday, April 27, 2010 4:40 AM
> To: java-user@lucene.apache.org
> Subject: Re: IndexWriter and memory usage
>
> Oooh -- I suspect you are hitting this issue:
>
&
:28 PM, Woolf, Ross wrote:
>> How do I get to the 2.9.x branch? Every link I take from the Lucene site
>> takes me to the trunk which I assume is the 3.x version. I've tried to look
>> around svn but can't find anything labeled 2.9.x. Is there a daily build of
&
s things down, so we don't want to run like this,
but we wanted to test the behavior if we did so).
Thanks,
Ross
-Original Message-
From: Michael McCandless [mailto:luc...@mikemccandless.com]
Sent: Wednesday, April 14, 2010 2:52 PM
To: java-user@lucene.apache.org
Subject:
the fix you
> put into it, but I'm not sure where I get it from.
>
> -Original Message-
> From: Michael McCandless [mailto:luc...@mikemccandless.com]
> Sent: Wednesday, April 14, 2010 4:12 AM
> To: java-user@lucene.apache.org
> Subject: Re: IndexWriter and memory
ry out the fix you put into it,
but I'm not sure where I get it from.
-Original Message-
From: Michael McCandless [mailto:luc...@mikemccandless.com]
Sent: Wednesday, April 14, 2010 4:12 AM
To: java-user@lucene.apache.org
Subject: Re: IndexWriter and memory usage
It looks like t
sage-
> From: Woolf, Ross [mailto:ross_wo...@bmc.com]
> Sent: Tuesday, April 13, 2010 1:29 PM
> To: java-user@lucene.apache.org
> Subject: RE: IndexWriter and memory usage
>
> Are these fixes in 2.9x branch? We are using 2.9x and can't move to 3x just
> yet. If so,
apache.org
Subject: RE: IndexWriter and memory usage
Are these fixes in 2.9x branch? We are using 2.9x and can't move to 3x just
yet. If so, where do I specifically pick this up from?
-Original Message-
From: Lance Norskog [mailto:goks...@gmail.com]
Sent: Monday, April 12, 2010 10
ccandless.com]
> Sent: Saturday, April 03, 2010 3:39 AM
> To: java-user@lucene.apache.org
> Subject: Re: IndexWriter and memory usage
>
> Hmm why is the heap dump so immense? Normally it contains the top N
> (eg 100) object types and their count/aggregate RAM usage.
>
> Ca
attachment). I'll see what I can do about reducing the
>> heap dump (It was supplied by a colleague).
>>
>>
>> -Original Message-
>> From: Michael McCandless [mailto:luc...@mikemccandless.com]
>> Sent: Saturday, April 03, 2010 3:39 AM
>> To
supplied by a colleague).
>
>
> -Original Message-
> From: Michael McCandless [mailto:luc...@mikemccandless.com]
> Sent: Saturday, April 03, 2010 3:39 AM
> To: java-user@lucene.apache.org
> Subject: Re: IndexWriter and memory usage
>
> Hmm why is the heap dump so immense?
urday, April 03, 2010 3:39 AM
> To: java-user@lucene.apache.org
> Subject: Re: IndexWriter and memory usage
>
> Hmm why is the heap dump so immense? Normally it contains the top N
> (eg 100) object types and their count/aggregate RAM usage.
>
> Can you attach the infoStream
To: java-user@lucene.apache.org
> Subject: Re: IndexWriter and memory usage
>
> Hmm, not good. Can you post a heap dump? Also, can you turn on
> infoStream, index up to the OOM @ 512 MB, and post the output?
>
> IndexWriter should not hang onto much beyond the RAM buffer.
5:21 PM
To: java-user@lucene.apache.org
Subject: Re: IndexWriter and memory usage
Hmm, not good. Can you post a heap dump? Also, can you turn on
infoStream, index up to the OOM @ 512 MB, and post the output?
IndexWriter should not hang onto much beyond the RAM buffer. But, it
does allocate and
Hmm, not good. Can you post a heap dump? Also, can you turn on
infoStream, index up to the OOM @ 512 MB, and post the output?
IndexWriter should not hang onto much beyond the RAM buffer. But, it
does allocate and then recycle this RAM buffer, so even in an idle
state (having indexed enough docs
19 matches
Mail list logo
