I was able to check your email just today. The agregegate is async,
but only process one input CAS at the same time,default numberOfCASes.
I read your possible explanation but i have no idea that another
thread can modificate the cas, because the last annotator's execution
is correct and only missing that the framework uima-as serializes the
cas.
This is the configuration of deploy of the aggregate:
<?xml version="1.0" encoding="UTF-8"?>
<analysisEngineDeploymentDescription
xmlns="http://uima.apache.org/resourceSpecifier";>
<name>XClusterAnalyzerAE Deploy Descriptor</name>
<description>Deploys XClusterAnalyzerAE</description>
<deployment protocol="jms" provider="activemq">
<service>
<inputQueue endpoint="XClusterAnalyzerAggregate"
brokerURL="${defaultBrokerURL}"/>
<topDescriptor>
<import location="./XClusterAnalyzerAggregate.xml"/>
</topDescriptor>
<!-- remoteReplyQueueScaleout for remote delegate-->
<analysisEngine inputQueueScaleout="2"
internalReplyQueueScaleout="3">
<delegates>
<analysisEngine key="FileSystemMultiplerCas">
<casMultiplier poolSize="10"/>
</analysisEngine>
<analysisEngine key="XFileFormatDetector">
<scaleout numberOfInstances="2"/>
<asyncAggregateErrorConfiguration>
<processCasErrors maxRetries="0"
continueOnRetryFailure="true"/>
</asyncAggregateErrorConfiguration>
</analysisEngine>
<analysisEngine key="XDataFileExtractor">
<scaleout numberOfInstances="2"/>
<asyncAggregateErrorConfiguration>
<processCasErrors maxRetries="0"
continueOnRetryFailure="true"/>
</asyncAggregateErrorConfiguration>
</analysisEngine>
<remoteAnalysisEngine key="XLanguageDetector">
<inputQueue endpoint="XLanguageDetector"
brokerURL="${defaultBrokerURL}"/>
<serializer method="xmi"/>
<asyncAggregateErrorConfiguration>
<processCasErrors maxRetries="0"
continueOnRetryFailure="true"/>
</asyncAggregateErrorConfiguration>
</remoteAnalysisEngine>
<analysisEngine key="XTokenizer">
<scaleout numberOfInstances="2"/>
<asyncAggregateErrorConfiguration>
<processCasErrors maxRetries="0"
continueOnRetryFailure="true"/>
</asyncAggregateErrorConfiguration>
</analysisEngine>
<analysisEngine key="XBoTModeler">
<scaleout numberOfInstances="3"/>
<asyncAggregateErrorConfiguration>
<processCasErrors maxRetries="0"
continueOnRetryFailure="true"/>
</asyncAggregateErrorConfiguration>
</analysisEngine>
<analysisEngine key="MergerInViewCasMultipler">
<casMultiplier poolSize="1"/>
</analysisEngine>
<analysisEngine key="XClusterAnalyzer">
<scaleout numberOfInstances="1"/>
<asyncAggregateErrorConfiguration>
<processCasErrors maxRetries="0"
continueOnRetryFailure="true"/>
</asyncAggregateErrorConfiguration>
</analysisEngine>
</delegates>
</analysisEngine>
</service>
</deployment>
</analysisEngineDeploymentDescription>
2017-02-10 16:43 GMT-05:00, Jaroslaw Cwiklik <uim...@gmail.com>:
> Just a bit more evidence. The caller of the gerSofaAddr()
>
> public void writeViewsCommons() throws Exception {
> // Get indexes for each SofaFS in the CAS
> int numViews = cas.getBaseSofaCount();
>
> for (int sofaNum = 1; sofaNum <= numViews; sofaNum++) {
> FSIndexRepositoryImpl loopIR = (FSIndexRepositoryImpl)
> cas.getBaseCAS().getSofaIndexRepository(sofaNum);
> final int sofaAddr = getSofaAddr(sofaNum);
>
> Not an expert of this code, but it smells like another thread is changing a
> CAS which is being serialized.
>
> -jerry
>
> On Fri, Feb 10, 2017 at 4:31 PM, Jaroslaw Cwiklik <uim...@gmail.com> wrote:
>
>> Is this a primitive (single-threaded) aggregate or async
>> (multi-threaded)?
>> If async, try to simplify and run primitive aggregate with scaleout=1.
>>
>> The CAS does not seem to be null in this case. The caller of the
>> getSerializedCas()
>> checks for null.
>>
>> The code dies here:
>> Caused by: java.lang.NullPointerException
>> at org.apache.uima.cas.impl.CasSerializerSupport$CasDocSerializ
>> er.getSofaAddr(CasSerializerSupport.java:454)
>>
>> public int getSofaAddr(int sofaNum) {
>> if (sofaNum != 1 || cas.isInitialSofaCreated()) { //skip if initial
>> view && no Sofa yet
>> // all
>> non-initial-views must have a sofa
>> * return ((CASImpl)cas.getView(sofaNum)).getSofaRef();*
>> }
>> return 0;
>> }
>>
>> Looks to me that getView(sofaNum) is returning null. Is it possible that
>> two threads are operating on the same CAS maybe? One removing a view
>> while
>> another trying to serialize. Have no idea what else could it be.
>>
>> -jerry
>>
>>
>>
>> On Fri, Feb 10, 2017 at 8:45 AM, nelson rivera <nelsonriver...@gmail.com>
>> wrote:
>>
>>> Hi, The first thing I did was these tests,i made a simple test case
>>> that create a Cas with 17 views and then serialize using
>>> XmiCasSerializer.serialize(newJCas.getCas(), fis) and serializes
>>> correctly.
>>> Also i made other test, initialize the same AE but of local way with
>>> UIMA API and process the same input documents and the processing is
>>> correct and then serialize the CAS, without problem.
>>>
>>> The error is with AE deployed in uima-as and consuming it.
>>>
>>> 2017-02-09 17:30 GMT-05:00, Marshall Schor <m...@schor.com>:
>>> > one thing that would help track this down is a small isolated test
>>> > case.
>>> >
>>> > Do you think uima-as is needed? I'm wondering if a simple test case
>>> which
>>> > generated 17 views and then tried to serialize would show the
>>> > failure...
>>> >
>>> > If you could supply a small test case that showed the failure so we
>>> could
>>> > reproduce it, that would enable a rapid resolution.
>>> >
>>> > -Marshall
>>> >
>>> >
>>> > On 2/9/2017 3:58 PM, Marshall Schor wrote:
>>> >> The line throwing the null pointer exception is :
>>> >>
>>> >> cas.getView(sofaNum).getSofaRef()
>>> >>
>>> >> So the NPE is either the cas is null, or the getView(sofaNum) is
>>> returning
>>> >> null.
>>> >>
>>> >> I'm not sure what the best way is to debug this...
>>> >>
>>> >> -Marshall
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> On 2/9/2017 12:42 PM, nelson rivera wrote:
>>> >>> I have a aggregate service uima-as, at the end of aggregate the cas
>>> >>> to
>>> >>> return is composed of as many views as the number of input files,
>>> >>> each
>>> >>> view with annotations of processing.
>>> >>> With a number of input documents less than 15 the processing is
>>> >>> successful always,
>>> >>> but if the number of documents is greater than 15, i get a
>>> >>> NullPointerException at the aggregate service trying to serialize
>>> >>> the
>>> >>> cas, not in the processing of AE aggregate.
>>> >>> the logs of aggregate service:
>>> >>>
>>> >>> 11:51:38.815 - 42:
>>> >>> cu.datys.xinetica.uima.core.MergerInViewCasMultipler.hasNext(285):
>>> >>> INFO: HasNext false
>>> >>> 11:51:38.875 - 44:
>>> >>> org.apache.uima.uimacpp.UimacppAnalysisComponent.log(396): INFO: :
>>> >>> XClusterAnalyzer::process --- OK
>>> >>> 11:51:39.145 - 45:
>>> >>> org.apache.uima.aae.controller.AggregateAnalysisEngineContro
>>> ller_impl.replyToClient:
>>> >>> WARNING: Service: XClusterAnalyzerAggregate Runtime Exception
>>> >>> 11:51:39.145 - 45:
>>> >>> org.apache.uima.aae.controller.AggregateAnalysisEngineContro
>>> ller_impl.replyToClient:
>>> >>> WARNING:
>>> >>> org.apache.uima.aae.error.AsynchAEException:
>>> >>> org.apache.uima.UIMARuntimeException
>>> >>> at
>>> >>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.getSer
>>> ializedCas(JmsOutputChannel.java:1265)
>>> >>> at
>>> >>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.sendRe
>>> ply(JmsOutputChannel.java:800)
>>> >>> at
>>> >>> org.apache.uima.aae.controller.AggregateAnalysisEngineContro
>>> ller_impl.sendReplyToRemoteClient(AggregateAnalysisEngineCon
>>> troller_impl.java:2173)
>>> >>> at
>>> >>> org.apache.uima.aae.controller.AggregateAnalysisEngineContro
>>> ller_impl.replyToClient(AggregateAnalysisEngineController_impl.java:2342)
>>> >>> at
>>> >>> org.apache.uima.aae.controller.AggregateAnalysisEngineContro
>>> ller_impl.finalStep(AggregateAnalysisEngineController_impl.java:1862)
>>> >>> at
>>> >>> org.apache.uima.aae.controller.AggregateAnalysisEngineContro
>>> ller_impl.executeFlowStep(AggregateAnalysisEngineController_
>>> impl.java:2489)
>>> >>> at
>>> >>> org.apache.uima.aae.controller.AggregateAnalysisEngineContro
>>> ller_impl.process(AggregateAnalysisEngineController_impl.java:1271)
>>> >>> at
>>> >>> org.apache.uima.aae.handler.HandlerBase.invokeProcess(Handle
>>> rBase.java:118)
>>> >>> at
>>> >>> org.apache.uima.aae.handler.input.ProcessResponseHandler.can
>>> celTimerAndProcess(ProcessResponseHandler.java:117)
>>> >>> at
>>> >>> org.apache.uima.aae.handler.input.ProcessResponseHandler.han
>>> dleProcessResponseWithCASReference(ProcessResponseHandler.java:485)
>>> >>> at
>>> >>> org.apache.uima.aae.handler.input.ProcessResponseHandler.han
>>> dle(ProcessResponseHandler.java:767)
>>> >>> at
>>> >>> org.apache.uima.aae.handler.HandlerBase.delegate(HandlerBase
>>> .java:149)
>>> >>> at
>>> >>> org.apache.uima.aae.handler.input.ProcessRequestHandler_impl
>>> .handle(ProcessRequestHandler_impl.java:1113)
>>> >>> at
>>> >>> org.apache.uima.aae.spi.transport.vm.UimaVmMessageListener.o
>>> nMessage(UimaVmMessageListener.java:107)
>>> >>> at
>>> >>> org.apache.uima.aae.spi.transport.vm.UimaVmMessageDispatcher
>>> $1.run(UimaVmMessageDispatcher.java:70)
>>> >>> at
>>> >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>>> Executor.java:1145)
>>> >>> at
>>> >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>>> lExecutor.java:615)
>>> >>> at
>>> >>> org.apache.uima.aae.UimaAsThreadFactory$1.run(UimaAsThreadFa
>>> ctory.java:132)
>>> >>> at java.lang.Thread.run(Thread.java:745)
>>> >>> Caused by: org.apache.uima.UIMARuntimeException
>>> >>> at
>>> >>> org.apache.uima.cas.impl.XmiCasSerializer.serialize(XmiCasSe
>>> rializer.java:420)
>>> >>> at
>>> >>> org.apache.uima.cas.impl.XmiCasSerializer.serialize(XmiCasSe
>>> rializer.java:385)
>>> >>> at
>>> >>> org.apache.uima.aae.UimaSerializer.serializeCasToXmi(UimaSer
>>> ializer.java:145)
>>> >>> at
>>> >>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.serial
>>> izeCAS(JmsOutputChannel.java:251)
>>> >>> at
>>> >>> org.apache.uima.adapter.jms.activemq.JmsOutputChannel.getSer
>>> ializedCas(JmsOutputChannel.java:1250)
>>> >>> ... 18 more
>>> >>> Caused by: java.lang.NullPointerException
>>> >>> at
>>> >>> org.apache.uima.cas.impl.CasSerializerSupport$CasDocSerializ
>>> er.getSofaAddr(CasSerializerSupport.java:454)
>>> >>> at
>>> >>> org.apache.uima.cas.impl.CasSerializerSupport$CasDocSerializ
>>> er.writeViewsCommons(CasSerializerSupport.java:465)
>>> >>> at
>>> >>> org.apache.uima.cas.impl.XmiCasSerializer$XmiDocSerializer.
>>> writeViews(XmiCasSerializer.java:572)
>>> >>> at
>>> >>> org.apache.uima.cas.impl.CasSerializerSupport$CasDocSerializ
>>> er.serialize(CasSerializerSupport.java:441)
>>> >>> at
>>> >>> org.apache.uima.cas.impl.XmiCasSerializer.serialize(XmiCasSe
>>> rializer.java:415)
>>> >>> ... 22 more
>>> >>>
>>> >>
>>> >
>>> >
>>>
>>
>>
>
