Re: Solr 4.0 Alpha taking lot of CPU

2012-07-11 Thread Yonik Seeley 
On Wed, Jul 11, 2012 at 8:11 PM, Pavitar Singh  wrote:
> We upgraded to Solr 4.0 Alpha and our CPU usage shot off to 400%.In
> profiling we are getting following trace.

That could either be good or bad.  Higher CPU can mean higher
concurrency.  Have you benchmarked your indexing performance?

Example: going from 60 minutes for indexing and 200% average CPU usage
to 30 minutes at 400% CPU would generally be considered a good thing.

-Yonik
http://lucidimagination.com

>
>- *100.0%*  -
> *java.lang*.Thread.run
>   - *42 Collapsed methods
> (show)
>   *
>  - *98.0%*  - *org.apache.lucene.index*
>  
> .DocumentsWriter.updateDocument
> - *77.0%*  - *org.apache.lucene.index*
> 
> .DocumentsWriterPerThread.updateDocument
>- *76.0%*  - *org.apache.lucene.index*
>
> .DocFieldProcessor.processDocument
>   - *76.0%*  - *org.apache.lucene.index*
>
> .DocInverterPerField.processFields
>  - *36.0%*  - *org.apache.lucene.analysis.miscellaneous*
>
> .TrimFilter.incrementToken
> - *35.0%*  - *org.apache.lucene.analysis.core*
>
> .LowerCaseFilter.incrementToken
>- *17.0%*  - *org.apache.lucene.analysis.ngram*
>
> .NGramTokenFilter.incrementToken
>   - *9.2%*  - *org.apache.lucene.util*
>
> .AttributeSource.clearAttributes
>- *34.0%*  - *org.apache.lucene.index*
>
> .TermsHashPerField.add
> - *12.0%*  - *org.apache.lucene.index*
>
> .FreqProxTermsWriterPerField.addTerm
>- *11.0%*  - *org.apache.lucene.index*
>
> .FreqProxTermsWriterPerField.writeProx
>   - *6.4%*  - *org.apache.lucene.index*
>
> .TermsHashPerField.writeVInt
>- *11.0%*  - *
> org.apache.lucene.analysis.tokenattributes*
>
> .CharTermAttributeImpl.fillBytesRef
>  - *15.0%*  - *org.apache.lucene.index*
> 
> .DocumentsWriterFlushControl.obtainAndLock

Re: Solr 4.0 Alpha taking lot of CPU

2012-07-11 Thread Pavitar Singh 
In text format

100.0%  - java.lang.Thread.run
42 Collapsed methods (show)
98.0%  - org.apache.lucene.index.DocumentsWriter.updateDocument
77.0%  - org.apache.lucene.index.DocumentsWriterPerThread.updateDocument
76.0%  - org.apache.lucene.index.DocFieldProcessor.processDocument
76.0%  - org.apache.lucene.index.DocInverterPerField.processFields
36.0%  - org.apache.lucene.analysis.miscellaneous.TrimFilter.incrementToken
35.0%  - org.apache.lucene.analysis.core.LowerCaseFilter.incrementToken
17.0%  - org.apache.lucene.analysis.ngram.NGramTokenFilter.incrementToken
9.2%  - org.apache.lucene.util.AttributeSource.clearAttributes
34.0%  - org.apache.lucene.index.TermsHashPerField.add
12.0%  - org.apache.lucene.index.FreqProxTermsWriterPerField.addTerm
11.0%  - org.apache.lucene.index.FreqProxTermsWriterPerField.writeProx
6.4%  - org.apache.lucene.index.TermsHashPerField.writeVInt
11.0%  -
org.apache.lucene.analysis.tokenattributes.CharTermAttributeImpl.fillBytesRef
15.0%  - org.apache.lucene.index.DocumentsWriterFlushControl.obtainAndLock

On Thu, Jul 12, 2012 at 5:41 AM, Pavitar Singh  wrote:

> Hi,
>
> We upgraded to Solr 4.0 Alpha and our CPU usage shot off to 400%.In
> profiling we are getting following trace.
>
>
>- *100.0%*  - 
> *java.lang*.Thread.run
>   - *42 Collapsed methods 
> (show)
>   *
>  - *98.0%*  - *org.apache.lucene.index*
>  
> .DocumentsWriter.updateDocument
> - *77.0%*  - *org.apache.lucene.index*
> 
> .DocumentsWriterPerThread.updateDocument
>- *76.0%*  - *org.apache.lucene.index*
>
> .DocFieldProcessor.processDocument
>   - *76.0%*  - *org.apache.lucene.index*
>   
> .DocInverterPerField.processFields
>  - *36.0%*  - *
>  org.apache.lucene.analysis.miscellaneous*
>  
> .TrimFilter.incrementToken
> - *35.0%*  - *org.apache.lucene.analysis.core*
> 
> .LowerCaseFilter.incrementToken
>- *17.0%*  - *org.apache.lucene.analysis.ngram*
>
> .NGramTokenFilter.incrementToken
>   - *9.2%*  - *org.apache.lucene.util*
>   
> .AttributeSource.clearAttributes
>- *34.0%*  - *org.apache.lucene.index*
>  
> .TermsHashPerField.add
> - *12.0%*  - *org.apache.lucene.index*
> 
> .FreqProxTermsWriterPerField.addTerm
>- *11.0%*  - *org.apache.lucene.index*
>
> .FreqProxTermsWriterPerField.writeProx
>   - *6.4%*  - *org.apache.lucene.index*
>   
> .TermsHashPerField.writeVInt
>- *11.0%*  - *
> org.apache.lucene.analysis.tokenattributes*
> 
> .CharTermAttributeImpl.fillBytesRef
>  - *15.0%*  - *org.apache.lucene.index*
> 
> .DocumentsWriterFlushControl.obtainAndLock
>
>

Solr 4.0 Alpha taking lot of CPU

2012-07-11 Thread Pavitar Singh 
Hi,

We upgraded to Solr 4.0 Alpha and our CPU usage shot off to 400%.In
profiling we are getting following trace.


   - *100.0%*  -
*java.lang*.Thread.run
  - *42 Collapsed methods
(show)
  *
 - *98.0%*  - *org.apache.lucene.index*
 
.DocumentsWriter.updateDocument
- *77.0%*  - *org.apache.lucene.index*

.DocumentsWriterPerThread.updateDocument
   - *76.0%*  - *org.apache.lucene.index*
   
.DocFieldProcessor.processDocument
  - *76.0%*  - *org.apache.lucene.index*

.DocInverterPerField.processFields
 - *36.0%*  - *org.apache.lucene.analysis.miscellaneous*

.TrimFilter.incrementToken
- *35.0%*  - *org.apache.lucene.analysis.core*

.LowerCaseFilter.incrementToken
   - *17.0%*  - *org.apache.lucene.analysis.ngram*

.NGramTokenFilter.incrementToken
  - *9.2%*  - *org.apache.lucene.util*

.AttributeSource.clearAttributes
   - *34.0%*  - *org.apache.lucene.index*

.TermsHashPerField.add
- *12.0%*  - *org.apache.lucene.index*

.FreqProxTermsWriterPerField.addTerm
   - *11.0%*  - *org.apache.lucene.index*

.FreqProxTermsWriterPerField.writeProx
  - *6.4%*  - *org.apache.lucene.index*

.TermsHashPerField.writeVInt
   - *11.0%*  - *
org.apache.lucene.analysis.tokenattributes*

.CharTermAttributeImpl.fillBytesRef
 - *15.0%*  - *org.apache.lucene.index*

.DocumentsWriterFlushControl.obtainAndLock