Increase heap size using UIMA with Solr

2013-08-27 Thread Jun Ohtani 
Dear all,

My co-worker use UIMA with Solr.
We have two problems increasing heap size and cpu usage.

We have two question.

1. o.a.u.jcas.impl.JCasImpl.loadJCasClasses() call every requests.
  Using UIMA with Solr, we send to request using UIMAUpdateRequestProcessor , 
Solr(Lucene) create AnalysisEngine instance every requests.
  Maybe AnalysisEngine call loadJCasClasses() methods every request, and load 
CasType from classloader.
  Why does Solr(Lucene) not cache an Analysis Engine instance?

2. o.a.u.analysis_emgine.impl.AnalysisEngineManagementImpl.usedRootNames 
increase entry.
  The unique MBean name create  classname and suffix number.
  AnalysisEngineManagementImpl.setName() use HashSet to check unique MBean name.
  Is it better using HashMap(classname, counter) Instead of HashSet ?


Regards


Jun Ohtani

Increase heap size using UIMA with Solr

2013-08-27 Thread Jun Ohtani 
Dear all,

My co-worker use UIMA with Solr.
We have two problems increasing heap size and cpu usage.

We have two question.

1. o.a.u.jcas.impl.JCasImpl.loadJCasClasses() call every requests. 
  Using UIMA with Solr, we send to request using UIMAUpdateRequestProcessor , 
Solr(Lucene) create AnalysisEngine instance every requests.
  Maybe AnalysisEngine call loadJCasClasses() methods every request, and load 
CasType from classloader.
  Why does Solr(Lucene) not cache an Analysis Engine instance? 

2. o.a.u.analysis_emgine.impl.AnalysisEngineManagementImpl.usedRootNames 
increase entry.
  The unique MBean name create  classname and suffix number.
  AnalysisEngineManagementImpl.setName() use HashSet to check unique MBean name.
  Is it better using HashMap(classname, counter) Instead of HashSet ?
  

Regards


Jun Ohtani

Re: Increase heap size using UIMA with Solr

2013-08-27 Thread Tommaso Teofili 
Hi Jun,

2013/8/26 Jun Ohtani 

> Dear all,
>
> My co-worker use UIMA with Solr.
> We have two problems increasing heap size and cpu usage.
>
> We have two question.
>
> 1. o.a.u.jcas.impl.JCasImpl.loadJCasClasses() call every requests.
>  Using UIMA with Solr, we send to request using UIMAUpdateRequestProcessor
> , Solr(Lucene) create AnalysisEngine instance every requests.
>  Maybe AnalysisEngine call loadJCasClasses() methods every request, and
> load CasType from classloader.
>  Why does Solr(Lucene) not cache an Analysis Engine instance?
>

which version of Solr are you (and your coworkers) using?
In the first versions it used to cache the AnalysisEngine while the latest
one, if I recall correctly, only caches the resource description but that
can be improved / changed I think. However this should be addressed on the
Lucene / Solr side since that code is hosted there.


>
> 2. o.a.u.analysis_emgine.impl.AnalysisEngineManagementImpl.usedRootNames
> increase entry.
>  The unique MBean name create  classname and suffix number.
>  AnalysisEngineManagementImpl.setName() use HashSet to check unique MBean
> name.
>  Is it better using HashMap(classname, counter) Instead of HashSet ?
>

as far as I can see it shouldn't differ too much, unless you got evidence
of that.

My 2 cents,
Tommaso


>
>
> Regards
>
> 
> Jun Ohtani