Hi Shahid, This is related to CTAKES-151 ( https://issues.apache.org/jira/browse/CTAKES-151) where mentioned that LVG has bad multithreaded behavior. You could either try update LVG or place some locking around calls to MutateToString which probably affect performance.
2017-02-12 12:21 GMT+06:00 shahid ashraf <[email protected]>: > Hi Sean > > > I found the line which is causing the null pointer exception in the > ctakes-LVG.. > its > *String[] output = out.split("\\|");* > > *but issue seems * > > String out = lvgCmd.MutateToString(word); > > returns null > > This only happens when running the LVG via CAS pool. > > It seems it has been patched here ... > > https://github.com/meikeric/ytex/blob/master/workspace/ > ctakes-patches/src/main/java/edu/mayo/bmi/uima/lvg/ae/LvgAnnotator.java > . > > My Question is why this happens in CAS pool (where each CAS has its own > resource.) > > and seems it happens on even ... normal words ... but when testing these > words one by one on same code String out = lvgCmd.MutateToString(word); it > returns back the canonical form > > > Here is the log when > > String out = lvgCmd.MutateToString(word) *FAILS* > > WARN LvgAnnotator - mutateToString returned null for: anemia > > 12 Feb 2017 11:20:32 WARN LvgAnnotator - mutateToString returned null for: > remained > > 12 Feb 2017 11:20:28 WARN LvgAnnotator - mutateToString returned null for: > course > > 12 Feb 2017 11:20:28 WARN LvgAnnotator - mutateToString returned null for: > profile. > > 12 Feb 2017 11:20:28 WARN LvgAnnotator - mutateToString returned null for: > a > > 12 Feb 2017 11:20:25 WARN LvgAnnotator - mutateToString returned null for: > pre-specified > > 12 Feb 2017 11:20:25 WARN LvgAnnotator - mutateToString returned null for: > these > > 12 Feb 2017 11:20:25 WARN LvgAnnotator - mutateToString returned null for: > inhibitors > > 12 Feb 2017 11:20:22 WARN LvgAnnotator - mutateToString returned null for: > Many > > 12 Feb 2017 11:20:20 WARN LvgAnnotator - mutateToString returned null for: > if > > 12 Feb 2017 11:20:15 WARN LvgAnnotator - mutateToString returned null for: > third > > 12 Feb 2017 11:20:12 WARN LvgAnnotator - mutateToString returned null for: > Prolonged > > When trying these on same code ... but single thread ... we are getting > results from LVG... > > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - process(JCas) > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - annotate over full doc text > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - apply LVG processing to get > canonical form > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - canonicalForm == null: > anemia|anemia|1|1|l+b|1| > > anemia|anemia|2|1|l+b|1| > > anemia|anemia|128|1|l+b|1| > > anemia|anemia|1024|1|l+b|1| > > anemia|anemion|128|512|l+b|1| > > anemia|anemium|128|512|l+b|1| > > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - apply LVG processing to get > canonical form > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - canonicalForm == null: > placebo|placebo|1|1|l+b|1| > > placebo|placebo|2|1|l+b|1| > > placebo|placebo|128|1|l+b|1| > > placebo|placebo|1024|1|l+b|1| > > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - apply LVG processing to get > canonical form > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - canonicalForm == null: > Prolonged|prolong|1024|1024|l+b|1| > > Prolonged|prolonge|1024|1024|l+b|1| > > Prolonged|prolonged|1|1|l+b|1| > > Prolonged|prolonged|2|1|l+b|1| > > Prolonged|prolonged|128|1|l+b|1| > > Prolonged|prolonged|1024|1|l+b|1| > > Prolonged|prolongeed|1024|1024|l+b|1| > > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - apply LVG processing to get > canonical form > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - canonicalForm == null: > third|third|1|1|l+b|1| > > third|third|2|1|l+b|1| > > third|third|128|1|l+b|1| > > third|third|1024|1|l+b|1| > > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - apply LVG processing to get > canonical form > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - canonicalForm == null: > inhibitors|inhibitor|128|512|l+b|1| > > inhibitors|inhibitor|1024|1024|l+b|1| > > inhibitors|inhibitors|1|1|l+b|1| > > inhibitors|inhibitors|2|1|l+b|1| > > inhibitors|inhibitors|128|1|l+b|1| > > inhibitors|inhibitors|1024|1|l+b|1| > > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - apply LVG processing to get > canonical form > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - canonicalForm == null: > these|these|1|1|l+b|1| > > these|these|2|1|l+b|1| > > these|these|128|1|l+b|1| > > these|these|1024|1|l+b|1| > > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - apply LVG processing to get > canonical form > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - canonicalForm == null: > pre-specified|pre-specifi|1024|1024|l+b|1| > > pre-specified|pre-specifie|1024|1024|l+b|1| > > pre-specified|pre-specified|1|1|l+b|1| > > pre-specified|pre-specified|2|1|l+b|1| > > pre-specified|pre-specified|128|1|l+b|1| > > pre-specified|pre-specified|1024|1|l+b|1| > > pre-specified|pre-specifieed|1024|1024|l+b|1| > > pre-specified|pre-specify|1024|1024|l+b|1| > > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - apply LVG processing to get > canonical form > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - canonicalForm == null: > profile.|profile.|1|1|l+b|1| > > profile.|profile.|2|1|l+b|1| > > profile.|profile.|128|1|l+b|1| > > profile.|profile.|1024|1|l+b|1| > > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - apply LVG processing to get > canonical form > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - canonicalForm == null: > course|course|1|1|l+b|1| > > course|course|2|1|l+b|1| > > course|course|128|1|l+b|1| > > course|course|1024|1|l+b|1| > > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - apply LVG processing to get > canonical form > > 12 Feb 2017 11:49:32 INFO LvgAnnotator - canonicalForm == null: > remained|remain|1024|1024|l+b|1| > > remained|remaine|1024|1024|l+b|1| > > remained|remained|1|1|l+b|1| > > remained|remained|2|1|l+b|1| > > remained|remained|128|1|l+b|1| > > remained|remained|1024|1|l+b|1| > > remained|remaineed|1024|1024|l+b|1| > > > > > On Sat, Feb 11, 2017 at 2:51 AM, shahid ashraf <[email protected]> wrote: > > > To add more it only happens when using caspool > > > > On Sat, Feb 11, 2017 at 2:48 AM, Finan, Sean < > > [email protected]> wrote: > > > >> Yes, thanks. I will try to look into this a little later. > >> > >> > >> > >> Sean > >> > >> > >> > >> *From:* shahid ashraf [mailto:[email protected]] > >> *Sent:* Friday, February 10, 2017 3:45 PM > >> *To:* Finan, Sean > >> *Subject:* Re: CAS Pool > >> > >> > >> > >> Are you looking for these lines > >> > >> > >> > >> 11 Feb 2017 02:13:59 INFO POSTagger - POS tagger model file: > >> org/apache/ctakes/postagger/models/mayo-pos.zip > >> > >> 11 Feb 2017 02:14:06 INFO LvgCmdApiResourceImpl - Loading NLM Norm and > >> Lvg with config file = /Users/shahid/projects/healthn > >> lp/examples/ctakes-web-client/target/classes/org/apache/ > >> ctakes/lvg/data/config/lvg.properties > >> > >> 11 Feb 2017 02:14:06 INFO LvgCmdApiResourceImpl - config file > absolute > >> path = /Users/shahid/projects/healthnlp/examples/ctakes-web-client/ > >> target/classes/org/apache/ctakes/lvg/data/config/lvg.properties > >> > >> 11 Feb 2017 02:14:06 INFO LvgCmdApiResourceImpl - cwd = > >> /Users/shahid/projects/healthnlp/examples/ctakes-web-client > >> > >> 11 Feb 2017 02:14:06 INFO LvgCmdApiResourceImpl - cd > >> /Users/shahid/projects/healthnlp/examples/ctakes-web-client/ > >> target/classes/org/apache/ctakes/lvg/ > >> > >> 11 Feb 2017 02:14:06 INFO LvgCmdApiResourceImpl - cd > >> /Users/shahid/projects/healthnlp/examples/ctakes-web-client > >> > >> 11 Feb 2017 02:14:07 INFO DrugMentionAnnotator - Finite state machines > >> loaded. > >> > >> > >> > >> On Sat, Feb 11, 2017 at 2:01 AM, Finan, Sean < > >> [email protected]> wrote: > >> > >> Hi Shahid, > >> > >> > >> > >> Are you getting any log information from the lvg classes during > >> initialization? > >> > >> > >> > >> Sean > >> > >> > >> > >> *From:* shahid ashraf [mailto:[email protected]] > >> *Sent:* Friday, February 10, 2017 2:50 PM > >> *To:* [email protected]; Finan, Sean > >> *Subject:* CAS Pool > >> > >> > >> > >> > >> Hi > >> > >> > >> > >> I was able to create the caspool using the descriptor approach, but > >> instead of getting ctakes blocked which was happening without caspool > when > >> multiple threads are ran. > >> > >> Currently i am getting following error: > >> > >> > >> > >> Feb 10, 2017 8:44:06 PM org.apache.uima.analysis_engin > >> e.impl.PrimitiveAnalysisEngine_impl callAnalysisComponentProcess(430) > >> > >> SEVERE: Exception occurred > >> > >> org.apache.uima.analysis_engine.AnalysisEngineProcessException > >> > >> at org.apache.ctakes.lvg.ae > >> <https://urldefense.proofpoint.com/v2/url?u=http- > 3A__org.apache.ctakes.lvg.ae&d=DwMFaQ&c=qS4goWBT7poplM69zy_ > 3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m= > ZDqfnqc8s6SO7UnXkW_LfiKDd3-1jgYMVLoLMdCrVC8&s= > mz9hN1V1RYm97GOnmPU3P52EoPoye6lBvN6AhIFsTL8&e=> > >> .LvgAnnotator.process(LvgAnnotator.java:310) > >> > >> at org.apache.uima.analysis_component.JCasAnnotator_ > ImplBase. > >> process(JCasAnnotator_ImplBase.java:48) > >> > >> at org.apache.uima.analysis_engin > >> e.impl.PrimitiveAnalysisEngine_impl.callAnalysisComponentPro > >> cess(PrimitiveAnalysisEngine_impl.java:396) > >> > >> at org.apache.uima.analysis_engin > >> e.impl.PrimitiveAnalysisEngine_impl.processAndOutputNewCASes > >> (PrimitiveAnalysisEngine_impl.java:314) > >> > >> at org.apache.uima.analysis_engin > >> e.asb.impl.ASB_impl$AggregateCasIterator.processUntilNextOut > >> putCas(ASB_impl.java:570) > >> > >> at org.apache.uima.analysis_engin > >> e.asb.impl.ASB_impl$AggregateCasIterator.<init>(ASB_impl.java:412) > >> > >> at org.apache.uima.analysis_engine.asb.impl.ASB_impl. > process( > >> ASB_impl.java:344) > >> > >> at org.apache.uima.analysis_engin > >> e.impl.AggregateAnalysisEngine_impl.processAndOutputNewCASes > >> (AggregateAnalysisEngine_impl.java:265) > >> > >> at org.apache.uima.analysis_engin > >> e.impl.AnalysisEngineImplBase.process(AnalysisEngineImplBase.java:269) > >> > >> at org.apache.uima.analysis_engin > >> e.impl.MultiprocessingAnalysisEngine_impl.process(Multiproce > >> ssingAnalysisEngine_impl.java:199) > >> > >> at org.apache.ctakes.web.client.s > >> ervlet.DemoServlet.doPost(DemoServlet.java:135) > >> > >> at javax.servlet.http.HttpServlet > >> .service(HttpServlet.java:755) > >> > >> at javax.servlet.http.HttpServlet > >> .service(HttpServlet.java:848) > >> > >> at org.eclipse.jetty.servlet.Serv > >> letHolder.handle(ServletHolder.java:684) > >> > >> at org.eclipse.jetty.servlet.Serv > >> letHandler.doHandle(ServletHandler.java:501) > >> > >> at org.eclipse.jetty.server.handl > >> er.ScopedHandler.handle(ScopedHandler.java:137) > >> > >> at org.eclipse.jetty.security.Sec > >> urityHandler.handle(SecurityHandler.java:557) > >> > >> at org.eclipse.jetty.server.session.SessionHandler. > doHandle( > >> SessionHandler.java:231) > >> > >> at org.eclipse.jetty.server.handler.ContextHandler. > doHandle( > >> ContextHandler.java:1086) > >> > >> at org.eclipse.jetty.servlet.Serv > >> letHandler.doScope(ServletHandler.java:428) > >> > >> at org.eclipse.jetty.server.session.SessionHandler.doScope( > >> SessionHandler.java:193) > >> > >> at org.eclipse.jetty.server.handler.ContextHandler.doScope( > >> ContextHandler.java:1020) > >> > >> at org.eclipse.jetty.server.handl > >> er.ScopedHandler.handle(ScopedHandler.java:135) > >> > >> at org.eclipse.jetty.server.handl > >> er.HandlerWrapper.handle(HandlerWrapper.java:116) > >> > >> at org.eclipse.jetty.server.Server.handle(Server.java:370) > >> > >> at org.eclipse.jetty.server.Abstr > >> actHttpConnection.handleRequest(AbstractHttpConnection.java:494) > >> > >> at org.eclipse.jetty.server.Abstr > >> actHttpConnection.content(AbstractHttpConnection.java:982) > >> > >> at org.eclipse.jetty.server.Abstr > >> actHttpConnection$RequestHandler.content(AbstractHttpConnect > >> ion.java:1043) > >> > >> at org.eclipse.jetty.http.HttpPar > >> ser.parseNext(HttpParser.java:865) > >> > >> at org.eclipse.jetty.http.HttpPar > >> ser.parseAvailable(HttpParser.java:240) > >> > >> at org.eclipse.jetty.server.Async > >> HttpConnection.handle(AsyncHttpConnection.java:82) > >> > >> at org.eclipse.jetty.io.nio.Selec > >> tChannelEndPoint.handle(SelectChannelEndPoint.java:667) > >> > >> at org.eclipse.jetty.io.nio.Selec > >> tChannelEndPoint$1.run(SelectChannelEndPoint.java:52) > >> > >> at org.eclipse.jetty.util.thread. > >> QueuedThreadPool.runJob(QueuedThreadPool.java:608) > >> > >> at org.eclipse.jetty.util.thread. > >> QueuedThreadPool$3.run(QueuedThreadPool.java:543) > >> > >> at java.lang.Thread.run(Thread.java:745) > >> > >> Caused by: org.apache.uima.analysis_engine. > AnalysisEngineProcessException > >> > >> at org.apache.ctakes.lvg.ae > >> <https://urldefense.proofpoint.com/v2/url?u=http- > 3A__org.apache.ctakes.lvg.ae&d=DwMFaQ&c=qS4goWBT7poplM69zy_ > 3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m= > ZDqfnqc8s6SO7UnXkW_LfiKDd3-1jgYMVLoLMdCrVC8&s= > mz9hN1V1RYm97GOnmPU3P52EoPoye6lBvN6AhIFsTL8&e=> > >> .LvgAnnotator.setCanonicalForm(LvgAnnotator.java:372) > >> > >> at org.apache.ctakes.lvg.ae > >> <https://urldefense.proofpoint.com/v2/url?u=http- > 3A__org.apache.ctakes.lvg.ae&d=DwMFaQ&c=qS4goWBT7poplM69zy_ > 3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m= > ZDqfnqc8s6SO7UnXkW_LfiKDd3-1jgYMVLoLMdCrVC8&s= > mz9hN1V1RYm97GOnmPU3P52EoPoye6lBvN6AhIFsTL8&e=> > >> .LvgAnnotator.annotateRange(LvgAnnotator.java:342) > >> > >> at org.apache.ctakes.lvg.ae > >> <https://urldefense.proofpoint.com/v2/url?u=http- > 3A__org.apache.ctakes.lvg.ae&d=DwMFaQ&c=qS4goWBT7poplM69zy_ > 3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m= > ZDqfnqc8s6SO7UnXkW_LfiKDd3-1jgYMVLoLMdCrVC8&s= > mz9hN1V1RYm97GOnmPU3P52EoPoye6lBvN6AhIFsTL8&e=> > >> .LvgAnnotator.process(LvgAnnotator.java:307) > >> > >> ... 35 more > >> > >> Caused by: java.lang.NullPointerException > >> > >> > >> > >> > >> > >> -- > >> > >> with Regards > >> > >> Shahid Ashraf > >> > >> > >> > >> > >> > >> -- > >> > >> with Regards > >> > >> Shahid Ashraf > >> > > > > > > > > -- > > with Regards > > Shahid Ashraf > > > > > > -- > with Regards > Shahid Ashraf >
