UmlsDictionaryLookupAnnotator" failed.

[samir chabou]

Hello,
I have set my umlsUser and umlsPassword, but I receive the error:Initialization 
of annotator class 
"org.apache.ctakes.dictionary.lookup.ae.UmlsDictionaryLookupAnnotator" failed


28 Oct 2018 16:46:43  INFO UmlsDictionaryLookupAnnotator - Using 
ctakes.umlsaddr: https://uts-ws.nlm.nih.gov/restful/isValidUMLSUser: 
samirchabouorg.apache.uima.resource.ResourceInitializationException: 
Initialization of annotator class 
"org.apache.ctakes.dictionary.lookup.ae.UmlsDictionaryLookupAnnotator" failed.  
(Descriptor: 
file:/C:/PicoWS/ctakes/ctakes-dictionary-lookup/desc/analysis_engine/DictionaryLookupAnnotatorUMLS.xml)
 ...

Note 1: I'm able to connect to the site UMLS Terminology Services -- Home

| 
| 
|  | 
UMLS Terminology Services -- Home


 |

 |

 |




using my umslUser and umlsPAssword
Note 2: I used to be able to connect to DictionaryLookupAnnotatorUMLS before my 
UMLS username expired.
Note 3: I'm using cTAKES v3.1

Thanks for your help in advance
Samir

Initialization of annotator class "org.apache.ctakes.dictionary.lookup.ae.UmlsDictionaryLookupAnnotator" failed

[samir chabou]

Hello,I have set my umlsUser and umlsPassword, but I receive the 
error:Initialization of annotator class 
"org.apache.ctakes.dictionary.lookup.ae.UmlsDictionaryLookupAnnotator" failed


28 Oct 2018 16:46:43  INFO UmlsDictionaryLookupAnnotator - Using 
ctakes.umlsaddr: https://uts-ws.nlm.nih.gov/restful/isValidUMLSUser: 
samirchabouorg.apache.uima.resource.ResourceInitializationException: 
Initialization of annotator class 
"org.apache.ctakes.dictionary.lookup.ae.UmlsDictionaryLookupAnnotator" failed.  
(Descriptor: 
file:/C:/PicoWS/ctakes/ctakes-dictionary-lookup/desc/analysis_engine/DictionaryLookupAnnotatorUMLS.xml)
 ...
Note: that I'm able to connect to the site UMLS Terminology Services -- Home
using my umslUser and umlsPAssword
Thanks for your helpSamir

Re: keep file name when using CPE_GUI

[samir chabou]

Pei,the problem that I used  the jCas to extract interrelated information as in 
he example bellow (EventMention by sentence):
   List tokens = JCasUtil.selectCovered(EventMention.class, 
sentence);
I'm not sure if I can do the same with FileWriterCasConsumer where it generates 
an XML file ?
 


 On Monday, June 15, 2015 4:15 PM, "Chen, Pei" 
 wrote:
   

 Samir,
I believe the
FileWriterCasConsumer.xml or FilesInDirectoryCasConsumer.example.xml 
Should use the DocumentID as the filename if it was populated.
--Pei
-Original Message-----
From: samir chabou [mailto:samir...@yahoo.com.INVALID] 
Sent: Monday, June 15, 2015 4:49 AM
To: dev@ctakes.apache.org; 
Subject: Re: keep file name when using CPE_GUI

Hi Pei,the CASconsumer is: XCAS Writer CAS ConsumerThanks 


    On Sunday, June 14, 2015 5:27 PM, "Chen, Pei" 
 wrote:
  

 Samir,
Which cas consumer are you using?

Sent from my iPhone

On Jun 14, 2015, at 11:24 AM, samir chabou 
mailto:samir...@yahoo.com>> wrote:

Hi,
When I use CPE_GUI it does not keep the input file name but it changes it to 
doc0.
Example input file = test123.txt
the output file for test123.txt from the CPE_GUI = doc0. Is there any way to 
get the output file name = test123 instead of doc0 ?
Thanks for your help

  


  

Re: keep file name when using CPE_GUI

[samir chabou]

Hi Pei,the CASconsumer is: XCAS Writer CAS ConsumerThanks 


 On Sunday, June 14, 2015 5:27 PM, "Chen, Pei" 
 wrote:
   

 Samir,
Which cas consumer are you using?

Sent from my iPhone

On Jun 14, 2015, at 11:24 AM, samir chabou 
mailto:samir...@yahoo.com>> wrote:

Hi,
When I use CPE_GUI it does not keep the input file name but it changes it to 
doc0.
Example input file = test123.txt
the output file for test123.txt from the CPE_GUI = doc0. Is there any way to 
get the output file name = test123 instead of doc0 ?
Thanks for your help

   

Re: keep file name when using CPE_GUI

[samir chabou]

Hi Pei,I'm using the UIMA_CPE_GUI--clinical_documents_pipeline.launche 


 On Sunday, June 14, 2015 5:27 PM, "Chen, Pei" 
 wrote:
   

 Samir,Which cas consumer are you using?

Sent from my iPhone
On Jun 14, 2015, at 11:24 AM, samir chabou  wrote:


Hi,When I use CPE_GUI it does not keep the input file name but it changes it to 
doc0.Example input file = test123.txtthe output file for test123.txt from the 
CPE_GUI = doc0. Is there any way to get the output file name = test123 instead 
of doc0 ?Thanks for your help


  

keep file name when using CPE_GUI

[samir chabou]

 Hi,When I use CPE_GUI it does not keep the input file name but it changes it 
to doc0.Example input file = test123.txtthe output file for test123.txt from 
the CPE_GUI = doc0. Is there any way to get the output file name = test123 
instead of doc0 ?Thanks for your help

Re: UMLS Authentication failing despite correct username and password

[samir chabou]

I got the same error.Please verify your UMLS license settings in the 
DictionaryLookupAnnotatorUMLS.xml configuration.
Although I'm able to connect to UMLS via the  website using my user name and 
password. Any idea ? 
Thanks  


 On Monday, May 11, 2015 5:44 PM, "Finan, Sean" 
 wrote:
   

 Hi Pedro,

Check the cTakesHsql.xml and make sure that the line matches:

https://uts-ws.nlm.nih.gov/restful/isValidUMLSUser"/>

In an older version of cTAKES with an output message as you have:
>>11 May 2015 15:59:47  INFO AbstractJCasTermAnnotator - Default - Loading 
>>dictionary into memory.  Initial run may take few mins to load. Please be 
>>patient...
That line got corrupted.

Sean

-Original Message-
From: Pedro Teixeira [mailto:teixeir...@gmail.com] 
Sent: Monday, May 11, 2015 5:30 PM
To: dev@ctakes.apache.org
Subject: UMLS Authentication failing despite correct username and password

So I've checked the Dictionary lookup XML file and that password works to log 
in via the website. This was also working last week but stopped at some point 
over the last week. I've got cTAKES running on a linux system so I can index 
batches of documents via a script. The exact error is as follows (with the 
username/password blocked out).

11 May 2015 15:59:26  INFO LvgCmdApiResourceImpl - cwd =
/home/PT/cTAKES/apache-ctakes-3.2.1
11 May 2015 15:59:26  INFO LvgCmdApiResourceImpl - cd 
/home/PT/cTAKES/apache-ctakes-3.2.1/resources/org/apache/ctakes/lvg/
11 May 2015 15:59:27  INFO LvgCmdApiResourceImpl - cd
/home/PT/cTAKES/apache-ctakes-3.2.1
11 May 2015 15:59:27  INFO ClearNLPDependencyParserAE - using Morphy analysis? 
true Loading configuration.
Loading feature templates.
Loading lexica.
Loading model:

11 May 2015 15:59:42  INFO Chunker - Chunker model file:
org/apache/ctakes/chunker/models/chunker-model.zip
11 May 2015 15:59:44  INFO ContextDependentTokenizerAnnotator - Finite state 
machines loaded.
11 May 2015 15:59:44  INFO ConstituencyParser - Initializing parser...
11 May 2015 15:59:46  INFO ContextAnnotator - SCOPE ORDER: [1, 3]
11 May 2015 15:59:46  INFO NegationContextAnalyzer - initBoundaryData() called 
for ContextInitializer
11 May 2015 15:59:47  INFO POSTagger - POS tagger model file:
org/apache/ctakes/postagger/models/mayo-pos.zip
11 May 2015 15:59:47  INFO AbstractJCasTermAnnotator - Default - Loading 
dictionary into memory.  Initial run may take few mins to load. Please be 
patient...
11 May 2015 15:59:47  INFO AbstractJCasTermAnnotator - Using dictionary lookup 
window type: org.apache.ctakes.typesystem.type.textspan.Sentence
11 May 2015 15:59:47  INFO AbstractJCasTermAnnotator - Exclusion tagset
loaded: CC CD DT EX IN LS MD PDT POS PP PP$ PRP PRP$ RP TO VB VBD VBG VBN VBP 
VBZ WDT WP WPS WRB
11 May 2015 15:59:47  INFO AbstractJCasTermAnnotator - Using minimum term text 
span: 3
11 May 2015 15:59:47  INFO DictionaryDescriptorParser - Parsing dictionary
specifications:
/home/PT/cTAKES/apache-ctakes-3.2.1/resources/org/apache/ctakes/dictionary/lookup/fast/cTakesHsql.xml
11 May 2015 15:59:48 ERROR UmlsUserApprover - UMLS Account at 
https://urldefense.proofpoint.com/v2/url?u=https-3A__uts-2Dws.nlm.nih.gov_restful_isValidUMLSUser&d=BQIBaQ&c=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU&r=fs67GvlGZstTpyIisCYNYmQCP6r0bcpKGd4f7d4gTao&m=oVzYGAl69NhMu6lQpKeatJrIGk2o_z2AZvjq7Z5J69g&s=_JNevHgYhyKm5PjIyFlYxIS1UWuR7J-n5V551hou2dM&e=
  is not valid for user # with ## Couldn't 
initialize processing engine.
  Initialization of CAS Processor with name 
"AggregatePlaintextFastUMLSProcessor" failed.


I also have a test implementation on a local windows 8 laptop that also fails 
now due to the same error so it seems like it's UMLS related issue but I 
haven't heard back from them yet and was hoping perhaps someone with cTAKES has 
previously experienced and resolved the issue.

Thanks!


  

Re: iterate on the features of CAS consumer (FileWriterCasConsumer)

[samir chabou]

Hi Tim,I was able to use CasIOUtil package to iterate on the Cas features. 
First, I need it to create a new Cas and I used JCasFactory for that. Below is 
the two lines of code. Thanks for your help

JCas jcas = JCasFactory.createJCas(); //create a new case 
CasIOUtil.readJCas(jcas, new File("C:\\temp\\uima\\xcas\\xCasAbstrct.xcas")); 
//load the existing Cas into the new one
Samir


 On Wednesday, April 15, 2015 2:53 PM, samir chabou  
wrote:
   

 Thanks Tim for your suggestion I'll try to experiment with the CasIOUtil 
method and keep the uesr/dev list posted.   


 On Wednesday, April 15, 2015 7:07 AM, "Miller, Timothy" 
 wrote:
   

 The standard way that we do save redundant processing time is by writing the 
CAS for each file to an XMI file after one pass on the data which runs all the 
analysis engines.

For example, if we are working on experiments, we have one pipeline that does 
all the NLP feature generation (POS tags, dependency parsing, dictionary 
lookup, etc.), and writes each document to an xmi file in a directory using 
UimaFit's CasIOUtil class:
https://uima.apache.org/d/uimafit-current/api/org/apache/uima/fit/util/CasIOUtil.html

Then in a second machine learning pipeline we read the xmi files (using a 
different CasIOUtil method) and vary any machine learning parameters we want 
using the same standard annotations.

Hope this helps.
Tim

________
From: samir chabou [samir...@yahoo.com.INVALID]
Sent: Monday, April 13, 2015 11:22 PM
To: dev@ctakes.apache.org; u...@ctakes.apache.org
Subject: Re: iterate on the features of CAS consumer (FileWriterCasConsumer)

  Hi,how can I load an existing FileWriterCasConsumer in a java code and 
iterate through the features in the FileWriterCasConsumer ?
Note: i was able to load the clinical pipeline in my java code and create a new 
jCas and process it; the problem with this is each time i ran the java code i 
have to reload the clinical pipeline which take a bit of time.
please advise Thanks


    On Saturday, April 11, 2015 12:54 AM, samir chabou  
wrote:


  Hi,how can I load an existing FileWriterCasConsumer in a java code and 
iterate through the features in the FileWriterCasConsumer ?
Note: i was able to load the clinical pipeline in my java code and create a new 
jCas and process it; the problem with this is each time i ran the java code i 
have to reload the clinical pipeline which take a bit of time.
Thanks



   

  

Re: iterate on the features of CAS consumer (FileWriterCasConsumer)

[samir chabou]

Thanks Tim for your suggestion I'll try to experiment with the CasIOUtil method 
and keep the uesr/dev list posted.   


 On Wednesday, April 15, 2015 7:07 AM, "Miller, Timothy" 
 wrote:
   

 The standard way that we do save redundant processing time is by writing the 
CAS for each file to an XMI file after one pass on the data which runs all the 
analysis engines.

For example, if we are working on experiments, we have one pipeline that does 
all the NLP feature generation (POS tags, dependency parsing, dictionary 
lookup, etc.), and writes each document to an xmi file in a directory using 
UimaFit's CasIOUtil class:
https://uima.apache.org/d/uimafit-current/api/org/apache/uima/fit/util/CasIOUtil.html

Then in a second machine learning pipeline we read the xmi files (using a 
different CasIOUtil method) and vary any machine learning parameters we want 
using the same standard annotations.

Hope this helps.
Tim

________
From: samir chabou [samir...@yahoo.com.INVALID]
Sent: Monday, April 13, 2015 11:22 PM
To: dev@ctakes.apache.org; u...@ctakes.apache.org
Subject: Re: iterate on the features of CAS consumer (FileWriterCasConsumer)

  Hi,how can I load an existing FileWriterCasConsumer in a java code and 
iterate through the features in the FileWriterCasConsumer ?
Note: i was able to load the clinical pipeline in my java code and create a new 
jCas and process it; the problem with this is each time i ran the java code i 
have to reload the clinical pipeline which take a bit of time.
please advise Thanks


    On Saturday, April 11, 2015 12:54 AM, samir chabou  
wrote:


  Hi,how can I load an existing FileWriterCasConsumer in a java code and 
iterate through the features in the FileWriterCasConsumer ?
Note: i was able to load the clinical pipeline in my java code and create a new 
jCas and process it; the problem with this is each time i ran the java code i 
have to reload the clinical pipeline which take a bit of time.
Thanks



  

Re: iterate on the features of CAS consumer (FileWriterCasConsumer)

[samir chabou]

I did reply to all in order to pick the dev and user list but apparently the 
reply all did not work.Thanks
 


 On Tuesday, April 14, 2015 8:47 PM, "Miller, Timothy" 
 wrote:
   

 OK, but it is better in the future if you just respond to the list -- that way 
everyone will be able to follow the thread of the conversation.Tim
From: samir chabou [samir...@yahoo.com]
Sent: Tuesday, April 14, 2015 6:09 PM
To: Miller, Timothy
Subject: Re: iterate on the features of CAS consumer (FileWriterCasConsumer)

Hi Timothy,That's correct.



On Tuesday, April 14, 2015 7:09 AM, "Miller, Timothy" 
 wrote:


Samir,
I'm not sure I understand your question. Are you saying you want to be able to 
look at/process annotations in a file more than once without re-processing the 
note?
Tim

________
From: samir chabou [samir...@yahoo.com.INVALID]
Sent: Monday, April 13, 2015 11:22 PM
To: dev@ctakes.apache.org;u...@ctakes.apache.org
Subject: Re: iterate on the features of CAS consumer (FileWriterCasConsumer)

  Hi,how can I load an existing FileWriterCasConsumer in a java code and 
iterate through the features in the FileWriterCasConsumer ?
Note: i was able to load the clinical pipeline in my java code and create a new 
jCas and process it; the problem with this is each time i ran the java code i 
have to reload the clinical pipeline which take a bit of time.
please advise Thanks


    On Saturday, April 11, 2015 12:54 AM, samir chabou  
wrote:


  Hi,how can I load an existing FileWriterCasConsumer in a java code and 
iterate through the features in the FileWriterCasConsumer ?
Note: i was able to load the clinical pipeline in my java code and create a new 
jCas and process it; the problem with this is each time i ran the java code i 
have to reload the clinical pipeline which take a bit of time.
Thanks





  

Re: iterate on the features of CAS consumer (FileWriterCasConsumer)

[samir chabou]

   Hi,how can I load an existing FileWriterCasConsumer in a java code and 
iterate through the features in the FileWriterCasConsumer ?
Note: i was able to load the clinical pipeline in my java code and create a new 
jCas and process it; the problem with this is each time i ran the java code i 
have to reload the clinical pipeline which take a bit of time.
please advise Thanks


 On Saturday, April 11, 2015 12:54 AM, samir chabou  
wrote:
   

   Hi,how can I load an existing FileWriterCasConsumer in a java code and 
iterate through the features in the FileWriterCasConsumer ?
Note: i was able to load the clinical pipeline in my java code and create a new 
jCas and process it; the problem with this is each time i ran the java code i 
have to reload the clinical pipeline which take a bit of time.
Thanks

  

iterate on the features of CAS consumer (FileWriterCasConsumer)

[samir chabou]

  Hi,how can I load an existing FileWriterCasConsumer in a java code and 
iterate through the features in the FileWriterCasConsumer ?
Note: i was able to load the clinical pipeline in my java code and create a new 
jCas and process it; the problem with this is each time i ran the java code i 
have to reload the clinical pipeline which take a bit of time.
Thanks

Re: error when running UIMA_CVD--clinical-document-pipeline

[samir chabou]

the heap problem is resolved.I craete a system variable 
variable name: _JAVA_OPTIONSvriable value: -Xmx512M
and in the UIMA_CVD--clinical_documents_pipeline arguments 
i put : -Xms512M -Xmx1g
 regards,Samir

 On Friday, November 28, 2014 9:00 PM, samir chabou  
wrote:
   

 when i try to run the UIMA_CVD--clinical-document-pipeline (and also the 
UIMA_CVD--POS_tagger) i got the error:

Error occurred during initialization of VM
Could not reserve enough space for 3145728KB object heap

i increase the vm variable for the heap to 3000 (-Xms3000M -Xmx3g) but i still 
get the error. I'm stacked with this error :) any idea ? thanks 
Samir 
 

   

error when running UIMA_CVD--clinical-document-pipeline

[samir chabou]

when i try to run the UIMA_CVD--clinical-document-pipeline (and also the 
UIMA_CVD--POS_tagger) i got the error:

Error occurred during initialization of VM
Could not reserve enough space for 3145728KB object heap

i increase the vm variable for the heap to 3000 (-Xms3000M -Xmx3g) but i still 
get the error. I'm stacked with this error :) any idea ? thanks 
Samir 
 

Re: cTakes Document Preprocessor Component Error

[samir chabou]

I'm trying to install cTAKES in eclipse but i got an error when i try a 
checkout as maven project from SCM.the error is:
',any help ?Thanks
 

 On Thursday, November 14, 2013 3:38 PM, "Masanz, James J." 
 wrote:
   

 Try adding

org.apache.ctakes.typesystem.types.TypeSystem

as the type system.

So the source looks like:







Hope that helps
-- James


From: user-return-336-Masanz.James=mayo@ctakes.apache.org 
[mailto:user-return-336-Masanz.James=mayo@ctakes.apache.org] On Behalf Of 
digital paula
Sent: Thursday, November 14, 2013 1:42 PM
To: u...@ctakes.apache.org
Subject: cTakes Document Preprocessor Component Error

Hello cTakes Community,
 
I'm back after hanging out with the UIMA guys :-) for a while, getting a better 
grasp of the UIMA framework that cTakes sits on.    I'm exploring and 
understanding cTakes components and starting with the Document Preprocessor:
https://cwiki.apache.org/confluence/display/CTAKES/cTAKES+3.0+-+Document+Preprocessor
 
I loaded the testpatient_cn_1.xml file in the CVD cTakes launcher and loaded 
the AggregateAE descriptor.   It loaded but when I ran it, I got the following 
error:    
 
Caused by: org.apache.uima.cas.CASRuntimeException: JCas type 
"org.apache.ctakes.typesystem.type.textspan.Segment" used in Java code, but was 
not declared in the XML type descriptor.
 
I looked in the AggregatorAE descriptor file along with it's dependency 
CdaCasInitializer.xml and see both have nothing in the Type System tab however 
I do see a handful of types specified on the Capabilities tab including 
"org.apache.ctakes.typesystem.type.textspan.Segment"
 
So I added Segment type to the Type System tab.  I reran and got this error:

11/14/13 2:23:39 PM - 14: 
org.apache.uima.tools.cvd.MainFrame.handleException(527): SEVERE: Feature "id" 
is not defined for type "org.apache.ctakes.typesystem.type.textspan.Segment".
org.apache.uima.analysis_engine.AnalysisEngineProcessException: Feature "id" is 
not defined for type "org.apache.ctakes.typesystem.type.textspan.Segment".
Is there a file descriptor already setup maybe in another install with the 
necessary types/features to run the Document Preprocessor or will I manually 
have to add all of them?   If not, I'd like to use UIMAfit to create the 
types/features to use with the Document Preprocessor as opposed to manually 
updating this descriptor with all the types and features needed.   Can someone 
advise?  
 
Thanks.
 
Regards,
Paula
 
 

 
 

   

Re: ebuging my own project under cTAKES

[samir chabou]

Hi James,
Good news James,  the source code for my project is now visible to Eclipse 
debugger. I linked the source of the uqoPico project to the starting project 
(for my case ctakes-clinical-pipeline project). the option of linking source is 
available when i use the java Perspective instead of the JavaEE Perspective.
Thanks for your e-mail it guided me to the right solution





On Wednesday, November 20, 2013 10:49:21 PM, samir chabou  
wrote:
 
Hi James,
Unfortunately no luck yet with my debug issue.

I'm using eclipse Version: Juno Service Release 2 (2013)
Do you do the compile within Eclipse or externally (such as from the command 
line) within Eclipse



When you say your project is at the same level as the other cTAKES 
projects, is that how it appears within Eclipse, or actually on the file
system? How can i check that the project UqoPico is in the file system ?

thanks Samir





On Monday, November 18, 2013 10:14:54 AM, "Masanz, James J." 
 wrote:

Hi Samir,

Any luck with this issue?
Are you using Eclipse? If so, which version?

If I understand correctly, the java build path for your project should include 
UQOPicoProject/src/main/java (or whatever directory you placed your source in)

Do you do the compile within Eclipse or externally (such as from the command 
line)

If I remember correctly, when Eclipse says that source is not available, it 
gives an option to click and link the source. That has worked for me to view 
source for things like jars.

When you say your project is at the same level as the other cTAKES projects, is 
that how it appears within Eclipse, or actually on the file system?

-- James

-Original Message-
From: dev-return-2216-Masanz.James=mayo@ctakes.apache.org 
[mailto:dev-return-2216-Masanz.James=mayo@ctakes.apache.org] On Behalf Of 
samir chabou
Sent: Thursday, November 14, 2013 10:30 PM
To: dev@ctakes.apache.org
Subject: ebuging my own project under cTAKES

Hi James and others
I create my own project UQOPicoProject under the same level than the other 
cTAKES's projects. When i try to debug UQOPicoProject the debugger can't stop 
on my break point and show a message "source not available". Any idea what 
would be the issue.
thanks
samir

Re: ebuging my own project under cTAKES

[samir chabou]

Hi James,
Unfortunately no luck yet with my debug issue.

I'm using eclipse Version: Juno Service Release 2 (2013)
Do you do the compile within Eclipse or externally (such as from the command 
line) within Eclipse



When you say your project is at the same level as the other cTAKES 
projects, is that how it appears within Eclipse, or actually on the file
 system? How can i check that the project UqoPico is in the file system ?

thanks Samir




On Monday, November 18, 2013 10:14:54 AM, "Masanz, James J." 
 wrote:
 
Hi Samir,

Any luck with this issue?
Are you using Eclipse? If so, which version?

If I understand correctly, the java build path for your project should include 
UQOPicoProject/src/main/java (or whatever directory you placed your source in)

Do you do the compile within Eclipse or externally (such as from the command 
line)

If I remember correctly, when Eclipse says that source is not available, it 
gives an option to click and link the source. That has worked for me to view 
source for things like jars.

When you say your project is at the same level as the other cTAKES projects, is 
that how it appears within Eclipse, or actually on the file system?

-- James

-Original Message-
From: dev-return-2216-Masanz.James=mayo@ctakes.apache.org 
[mailto:dev-return-2216-Masanz.James=mayo@ctakes.apache.org] On Behalf Of 
samir chabou
Sent: Thursday, November 14, 2013 10:30 PM
To: dev@ctakes.apache.org
Subject: ebuging my own project under cTAKES

Hi James and others
I create my own project UQOPicoProject under the same level than the other 
cTAKES's projects. When i try to debug UQOPicoProject the debugger can't stop 
on my break point and show a message "source not available". Any idea what 
would be the issue.
thanks
samir

ebuging my own project under cTAKES

[samir chabou]

Hi James and others
I create my own project UQOPicoProject under the same level than the other 
cTAKES's projects. When i try to debug UQOPicoProject the debugger can't stop 
on my break point and show a message "source not available". Any idea what 
would be the issue.
thanks
samir

Re: Synonymous terms

[samir chabou]

thanks James, definitively these explanations helps





On Wednesday, October 23, 2013 12:42:48 PM, "Masanz, James J." 
 wrote:
 

Maybe an example is best.

There is a term in  MeSH "Progeria-Like Syndrome"
The CUI for that is C0009207

That term (using those words) does not appear in SNOMED-CT.
But there is at least one SNOMED-CT term (Cockayne syndrome, with code 
21086008) associated with that CUI within the UMLS metathesaurus.
Therefore cTAKES will annotate "Progeria-Like Syndrome" with CUI C0009207and 
also cTAKES will list the SNOMED-CT codes that are associated with that CUI, 
even though none of the SNOMED-CT terms associated with the CUI C0009207 
contain the word "like".

However, the MeSH code (D003057) for "Progeria-Like Syndrome" will not be 
included within the CAS.

Hope that helps
-- James



-Original Message-
From: dev-return-2138-Masanz.James=mayo@ctakes.apache.org 
[mailto:dev-return-2138-Masanz.James=mayo@ctakes.apache.org] On Behalf Of 
samir chabou
Sent: Tuesday, October 22, 2013 6:55 PM
To: samir chabou; dev@ctakes.apache.org
Subject: Re: Synonymous terms 

sorry i'm having some problem with my e-mail
correct me if i'm wrong. Based in what is said, when i do a lookup i'll get the 
term (entity, event for example) that can be from MeSH, ICD9 or NCI but the 
ontologyConceptArray will contains the codes synonyms in SNOMED-CT  




On Tuesday, October 22, 2013 12:23:06 PM, "Masanz, James J." 
 wrote:

Just in case it wasn't clear, when I said it only contains SNOMED-CT codes, I 
pointing out that it doesn't have entries for ICD9 or MeSH or NCI Thesaurus, 
even though those are used when doing the dictionary lookup.  I wasn't thinking 
about the codes vs. text part.


-Original Message-
From: dev-return-2126-Masanz.James=mayo@ctakes.apache.org 
[mailto:dev-return-2126-Masanz.James=mayo@ctakes.apache.org] On Behalf Of 
Chen, Pei
Sent: Tuesday, October 22, 2013 11:17 AM
To: dev@ctakes.apache.org; 'samir chabou'
Subject: RE: Synonymous terms 

> if I need to get term name do I have to do a lookup on umls ?
FYI: I was proposing adding an additional attribute to store the 
description/preferredText(term) [1] since this information is already available 
in the dictionary lookup.
I think most folks would find this useful in additional to just saving the 
CUI/Code. Otherwise, they would have to do another lookup further downstream to 
get the description of the CUI/Code.

[1] https://issues.apache.org/jira/browse/CTAKES-224
--Pei

> -Original Message-
> From: Masanz, James J. [mailto:masanz.ja...@mayo.edu]
> Sent: Tuesday,
October 22, 2013 10:57 AM
> To: 'dev@ctakes.apache.org'; 'samir chabou'
> Subject: RE: Synonymous terms
> 
> 
> ontologyConceptArray only contains SNOMED-CT codes (except for
> medications, where it contains RxNorm codes or CUIs)
> 
> -Original Message-----
> From: dev-return-2119-Masanz.James=mayo@ctakes.apache.org
> [mailto:dev-return-2119-Masanz.James=mayo@ctakes.apache.org] On
> Behalf Of samir chabou
> Sent:
Monday, October 21, 2013 6:59 PM
> To: dev@ctakes.apache.org
> Subject: Fw: Synonymous terms
> 
> 
> 
> 
> 
> 
> On Monday, October 21, 2013 7:58:01 PM, samir chabou
>  wrote:
> 
> Synonymous
> terms
> 
> 
> 
> Given a C0001621 I need to get its synonyms terms for example in MeSH,
> ICD, SNOMED (as shown bellow)…Is that possible ?
> 
> 
> 
> Synonymous
> terms  Adrenal gland diseases     MeSH  D000307 Diseases of 
> the
> adrenal glands        SNOMED DB-7
>     ...
> 
>   Are  the
> synonyms kept in ontologyConceptArray ?  If yes, then in the
> ontologyConceptArray the information displayed is about identifier (code,
> cui, tui, oid …) if I need to get term name do I have to do a lookup on umls ?
> thanks
> Samir

Re: CTAKES-248- include original covered text of NEs which can't be recovered post if NE is from a disjoint span

[samir chabou]

hi Pei,
is this mean that your proposition bellow is now ready to use. 
<< FYI: I was proposing adding an additional attribute to store the 
description/preferredText(term) [1] since this information is already 
available in the dictionary lookup.
I think most folks 
would find this useful in additional to just saving the CUI/Code. 
Otherwise, they would have to do another lookup further downstream to 
get the description of the CUI/Code.>> 





On Tuesday, October 22, 2013 4:01:35 PM, "Chen, Pei" 
 wrote:
 
Done.


> -Original Message-
> From: Masanz, James J. [mailto:masanz.ja...@mayo.edu]
> Sent: Tuesday, October 22, 2013 2:33 PM
> To: 'dev@ctakes.apache.org'
> Subject: RE: CTAKES-248- include original covered text of NEs which can't be
> recovered post if NE is from a disjoint span
> 
> Sure, if you would, that would be great. Thanks.
> 
> -Original Message-
> From: dev-return-2128-Masanz.James=mayo@ctakes.apache.org
> [mailto:dev-return-2128-Masanz.James=mayo@ctakes.apache.org] On
> Behalf Of Chen, Pei
> Sent: Tuesday, October 22, 2013 1:30 PM
> To: dev@ctakes.apache.org
> Subject: RE: CTAKES-248- include original covered text of NEs which can't be
> recovered post if NE is from a disjoint span
> 
> James,
> I was making some changes to the ctakes common type system for CTAKES-
> 224 (Adding a field to save the UMLS term/text in addition to the
> CUI/Codes).
> Do you want me to also make originalText an FSArray instead of
> String while I have these files open?
> 
> --Pei
> 
> > -Original Message-
> > From: Chen, Pei [mailto:pei.c...@childrens.harvard.edu]
> > Sent: Wednesday, October 02, 2013 10:23 AM
> > To: dev@ctakes.apache.org
> > Subject: RE: CTAKES-248- include original covered text of NEs which
> > can't be recovered post if NE is from a disjoint span
> >
> > +1 to have a pointer back to the BaseToken(s) rather than a | String
> > +(so we
> > could get back the spans and other info if needed).
> > I think the atom will be slightly different, take for example:
> > Perhaps with an example:
> > Sentence/LookupWindow: "alcoholic liver disease was acute."
> > originalText: "disease acute" [New feature to store the Tokens that
> > were matched due to the permutations?]
> > UmlsConcept.cui: C0001314
> > UmlsConcept.preferredText: "Acute Disease" [New feature to store the
> > atom/text returned by the UMLS CUI]
> >
> > I also ran into a similar case where I wish
> > IdentifiedAnnotation.segmentID/SentenceID was the actual Segment type
> > and not a String.
> >
> > This is just my 2 cents... open to ideas though.
> > --Pei
> >
> >
> > > -Original Message-
> > > From: Richard Eckart de Castilho [mailto:richard.eck...@gmail.com]
> > > Sent: Wednesday, October 02, 2013 3:19 AM
> > > To: dev@ctakes.apache.org
> > > Subject: Re: CTAKES-248- include original covered text of NEs which
> > > can't be recovered post if NE is from a disjoint span
> > >
> > > What benefit would it have to store a string with some separation
> > > character (which may mean that the separation character in the
> > > elements may need to be escaped), over using a feature of type
> > > FSArray pointing to the original segments?
> > >
> > > Not sure if that is what Karthik meant when referring to fetching
> > > the matched atom.
> > >
> > > -- Richard
> > >
> > > On 02.10.2013, at 01:46, Karthik Sarma  wrote:
> > >
> > > > Hmm, couldn't you just fetch the matched atom and use that? Should
> > > > be the same information (without, I suppose, the original ordering
> > > > and
> > split).
> > > >
> > > > --
> > > > Karthik Sarma
> > > > UCLA Medical Scientist Training Program Class of 20??
> > > > Member, UCLA Medical Imaging & Informatics Lab Member, CA
> > Delegation
> > > > to the House of Delegates of the American Medical Association
> > > > ksa...@ksarma.com
> > > > gchat: ksa...@gmail.com
> > > > linkedin: www.linkedin.com/in/ksarma
> > > >
> > > >
> > > > On Tue, Oct 1, 2013 at 12:37 PM, Masanz, James J.
> > > wrote:
> > > >
> > > >> Yes, this would help address that multiple permutations example.
> > > >> The new getOriginalText method would return something like
> > > >> "Acute|Disease".  Right now I'm thinking of just using vertical
> > > >> bar as delimiter, to start with at least, but think it should be
> configurable.
> > > >>
> > > >> -Original Message-
> > > >> From: dev-return-2067-Masanz.James=mayo@ctakes.apache.org
> > > [mailto:
> > > >> dev-return-2067-Masanz.James=mayo@ctakes.apache.org] On
> > > Behalf Of
> > > >> Chen, Pei
> > > >> Sent: Tuesday, October 01, 2013 9:38 AM
> > > >> To: dev@ctakes.apache.org
> > > >> Subject: CTAKES-248- include original covered text of NEs which
> > > >> can't be recovered post if NE is from a disjoint span
> > > >>
> > > >> This sounds pretty cool.
> > > >> James, will this address the multiple permutations lookup example:
> > > >> "Acute alcoholic liver disease."  There is a cui: C0001314: Acute
> > > >> Disease, but if you g

Re: Synonymous terms

[samir chabou]

sorry i'm having some problem with my e-mail
correct me if i'm wrong. Based in what is said, when i do a lookup i'll get the 
term (entity, event for example) that can be from MeSH, ICD9 or NCI but the 
ontologyConceptArray will contains the codes synonyms in SNOMED-CT  




On Tuesday, October 22, 2013 12:23:06 PM, "Masanz, James J." 
 wrote:
 
Just in case it wasn't clear, when I said it only contains SNOMED-CT codes, I 
pointing out that it doesn't have entries for ICD9 or MeSH or NCI Thesaurus, 
even though those are used when doing the dictionary lookup.  I wasn't thinking 
about the codes vs. text part.


-Original Message-
From: dev-return-2126-Masanz.James=mayo@ctakes.apache.org 
[mailto:dev-return-2126-Masanz.James=mayo@ctakes.apache.org] On Behalf Of 
Chen, Pei
Sent: Tuesday, October 22, 2013 11:17 AM
To: dev@ctakes.apache.org; 'samir chabou'
Subject: RE: Synonymous terms 

> if I need to get term name do I have to do a lookup on umls ?
FYI: I was proposing adding an additional attribute to store the 
description/preferredText(term) [1] since this information is already available 
in the dictionary lookup.
I think most folks would find this useful in additional to just saving the 
CUI/Code. Otherwise, they would have to do another lookup further downstream to 
get the description of the CUI/Code.

[1] https://issues.apache.org/jira/browse/CTAKES-224
--Pei

> -Original Message-
> From: Masanz, James J. [mailto:masanz.ja...@mayo.edu]
> Sent: Tuesday,
 October 22, 2013 10:57 AM
> To: 'dev@ctakes.apache.org'; 'samir chabou'
> Subject: RE: Synonymous terms
> 
> 
> ontologyConceptArray only contains SNOMED-CT codes (except for
> medications, where it contains RxNorm codes or CUIs)
> 
> -Original Message-
> From: dev-return-2119-Masanz.James=mayo....@ctakes.apache.org
> [mailto:dev-return-2119-Masanz.James=mayo@ctakes.apache.org] On
> Behalf Of samir chabou
> Sent:
 Monday, October 21, 2013 6:59 PM
> To: dev@ctakes.apache.org
> Subject: Fw: Synonymous terms
> 
> 
> 
> 
> 
> 
> On Monday, October 21, 2013 7:58:01 PM, samir chabou
>  wrote:
> 
> Synonymous
> terms
> 
> 
> 
> Given a C0001621 I need to get its synonyms terms for example in MeSH,
> ICD, SNOMED (as shown bellow)…Is that possible ?
> 
> 
> 
> Synonymous
> terms  Adrenal gland diseases     MeSH  D000307 Diseases of 
> the
> adrenal glands        SNOMED DB-7
>     ...
> 
>   Are  the
> synonyms kept in ontologyConceptArray ?  If yes, then in the
> ontologyConceptArray the information displayed is about identifier (code,
> cui, tui, oid …) if I need to get term name do I have to do a lookup on umls ?
> thanks
> Samir

Re: Synonymous terms

[samir chabou]

thanks James for the info.,
do you know if there is a way in cTAKES to allow me getting the synonyms for a 
given CUI ?   or should i try to use an external API that could do that ?




On Tuesday, October 22, 2013 10:55:54 AM, "Masanz, James J." 
 wrote:
 

ontologyConceptArray only contains SNOMED-CT codes (except for medications, 
where it contains RxNorm codes or CUIs)


-Original Message-
From: dev-return-2119-Masanz.James=mayo@ctakes.apache.org 
[mailto:dev-return-2119-Masanz.James=mayo@ctakes.apache.org] On Behalf Of 
samir chabou
Sent: Monday, October 21, 2013 6:59 PM
To: dev@ctakes.apache.org
Subject: Fw: Synonymous terms 






On Monday, October 21, 2013 7:58:01 PM, samir chabou  wrote:

Synonymous
terms 
 
 

Given a C0001621 I need to get its synonyms terms for example
in MeSH, ICD, SNOMED (as shown bellow)…Is that possible ?   



Synonymous
terms  Adrenal gland diseases     MeSH  D000307
Diseases of the adrenal glands        SNOMED DB-7
    ...

  Are  the
synonyms kept in ontologyConceptArray ?  If yes, then in the 
ontologyConceptArray the
information displayed is about identifier (code, cui, tui, oid …) if I need to
get term name do I have to do a lookup on umls ?
thanks
Samir

Fw: Synonymous terms

[samir chabou]

On Monday, October 21, 2013 7:58:01 PM, samir chabou  wrote:
 
Synonymous
terms 
 
 

Given a C0001621 I need to get its synonyms terms for example
in MeSH, ICD, SNOMED (as shown bellow)…Is that possible ?   



Synonymous
terms  Adrenal gland diseases     MeSH  D000307
Diseases of the adrenal glands        SNOMED DB-7
    ...

  Are  the
synonyms kept in ontologyConceptArray ?  If yes, then in the 
ontologyConceptArray the
information displayed is about identifier (code, cui, tui, oid …) if I need to
get term name do I have to do a lookup on umls ?
thanks
Samir

Re: implementation of << affects>> relation

[samir chabou]

Hi Dima,
Ok, I understand the situation.
Thanks Samir





On Monday, October 14, 2013 11:36:18 AM, Dmitriy Dligach 
 wrote:
 
Hi Samir,

Unfortunately, we cannot release the SHARP corpus at this point. The 
ShARe corpus will be made available via SemEval 2014.

However, we were planning to retrain the relation system to include the 
remaining relation types. The models will be available as a part of cTAKES.

Thanks for your offer of help!

Dima


On 10/10/2013 03:49 PM, samir chabou wrote:
> Thanks Dmitriy for the article. I read it attentively and found it 
> veryinteresting.
>  
> In the conclusion of the article you have mentioned that your system is 
> easily extendable to other relationssuch as manages/treats (which is a sub 
> relation of affects).
>  
> As part of my learning curve I’ll be interested in doing the extension and 
> retrain your classifier to include manages/treats under your lead and 
> coaching. This task will be beneficial for our team to gain practical 
> experienceon how svm works,and at the same timeit shouldcontribute to enhance 
> the quality of the alignment relationbetween the medical question and the 
> PICO structure. Also, this experience will certainly boost my participation 
> inthe cTAKES community.
>  
> On the base of your paper I suggest the following approachfor addingthe 
> manages/treats relation:
>  
> ·        Training corpus: SHARP and ShARe (how couldI get a copy of them, if 
> possible)
>  
> ·        Set of features related to the entities ofthe manages/treats 
> relation:  thesame features that you are using for the LocationOf and 
> DegreeOf relations
>  
> ·        Classifier parameters: need to be identified (probablythe same 
> parameters asyours;  I don’t know yet how these parameters can be set)
>  
> ·        To retrain your (binary linear) classifier for manages/treats 
> relation: I need to know the tool used for this. Is it MALLET that you have 
> used or some thing else?
>  
> ·        Test corpus: sub set of the SHARP and ShARe in order to be able to 
> quantify the results (f
> score, precision and recall) since these corporaare already annotated
>  
> ·        To validate the results
>  
> ·        If validated, to deploy the manages/treats relation in 
> ctakes-relation-extractorReply,
>
>
>
> On Tuesday, October 8, 2013 9:31:28 AM, Dmitriy Dligach 
>  wrote:
>  
> Hi Samir,
>
> Before we talk any further, I'd like to mention that the relation
> extractor is a machine learning system. You can read more about it here:
>
> http://jamia.bmj.com/content/early/2013/10/03/amiajnl-2013-001766.full
>
> So if you want to extend it, you'll need some data that annotates the
> "affects" relation. Do you have this kind of data?
>
> Dima
>
>
> On 10/07/2013 05:38 PM, samir chabou wrote:
>> Hi Pei, Dimitry and any other that may help on
>> this …
>>
>   Currently I’m using cTAKES to semantically   analyze
>> a medical question and then try to map it to PICO structure based on 
>> different
>> medical type (relations, NE) identified in the question. I’m interested in 
>> extending
>> the ctakes-relation-extractor project in order to handle the << affects>> 
>> relation of the UMLS
>> semantic network.
>> First of all is that possible?
>> If yes,
>> what is the involvement?  My plan is:
>> 1)      To proceed with the implementation locally and
>> then check it in to be validated and deployed later (if it’s approved).
>> 2)      How ever I’m not very familiar with the subject
>> matter on how UMLS semantic relations are implemented in cTAKES. Is the <<
>   affects>>
>> relation will follow the same logic than LocationOfRelation  and 
>> DegreeOfLocation?  Which kind of IdentifiedAnnotation that the <<
>> affects>> relation will relates?  It
>> would be very appreciated if some one guides me on this matter.
>>    
>> Thanks Samir

-- 
Dmitriy Dligach, PhD
Research Fellow
Children's Hospital Informatics Program
Boston Children's Hospital and Harvard Medical School
(617) 919-3596

Re: implementation of << affects>> relation

[samir chabou]

Thanks Dmitriy for the article. I read it attentively and found it 
veryinteresting. 
 
In the conclusion of the article you have mentioned that your system is easily 
extendable to other relationssuch as manages/treats (which is a sub relation of 
affects).
 
As part of my learning curve I’ll be interested in doing the extension and 
retrain your classifier to include manages/treats under your lead and coaching. 
This task will be beneficial for our team to gain practical experienceon how 
svm works,and at the same timeit shouldcontribute to enhance the quality of the 
alignment relationbetween the medical question and the PICO structure. Also, 
this experience will certainly boost my participation inthe cTAKES community.
 
On the base of your paper I suggest the following approachfor addingthe 
manages/treats relation:
 
·    Training corpus: SHARP and ShARe (how couldI get a copy of them, if 
possible)
 
·    Set of features related to the entities ofthe manages/treats relation: 
 thesame features that you are using for the LocationOf and DegreeOf relations
 
·    Classifier parameters: need to be identified (probablythe same 
parameters asyours;  I don’t know yet how these parameters can be set)
 
·    To retrain your (binary linear) classifier for manages/treats 
relation: I need to know the tool used for this. Is it MALLET that you have 
used or some thing else?
 
·    Test corpus: sub set of the SHARP and ShARe in order to be able to 
quantify the results (f 
score, precision and recall) since these corporaare already annotated
 
·    To validate the results
 
·    If validated, to deploy the manages/treats relation in 
ctakes-relation-extractorReply, 



On Tuesday, October 8, 2013 9:31:28 AM, Dmitriy Dligach 
 wrote:
 
Hi Samir,

Before we talk any further, I'd like to mention that the relation 
extractor is a machine learning system. You can read more about it here:

http://jamia.bmj.com/content/early/2013/10/03/amiajnl-2013-001766.full

So if you want to extend it, you'll need some data that annotates the 
"affects" relation. Do you have this kind of data?

Dima


On 10/07/2013 05:38 PM, samir chabou wrote:
> Hi Pei, Dimitry and any other that may help on
> this …
>
 Currently I’m using cTAKES to semantically   analyze
> a medical question and then try to map it to PICO structure based on different
> medical type (relations, NE) identified in the question. I’m interested in 
> extending
> the ctakes-relation-extractor project in order to handle the << affects>> 
> relation of the UMLS
> semantic network.
> First of all is that possible?
> If yes,
> what is the involvement?  My plan is:
> 1)      To proceed with the implementation locally and
> then check it in to be validated and deployed later (if it’s approved).
> 2)      How ever I’m not very familiar with the subject
> matter on how UMLS semantic relations are implemented in cTAKES. Is the <<
 affects>>
> relation will follow the same logic than LocationOfRelation  and 
> DegreeOfLocation?  Which kind of IdentifiedAnnotation that the <<
> affects>> relation will relates?  It
> would be very appreciated if some one guides me on this matter.
>  
> Thanks Samir

-- 
Dmitriy Dligach, PhD
Research Fellow
Children's Hospital Informatics Program
Boston Children's Hospital and Harvard Medical School
(617) 919-3596

implementation of << affects>> relation

[samir chabou]

Hi Pei, Dimitry and any other that may help on
this …
Currently I’m using cTAKES to semantically   analyze
a medical question and then try to map it to PICO structure based on different
medical type (relations, NE) identified in the question. I’m interested in 
extending
the ctakes-relation-extractor project in order to handle the << affects>> 
relation of the UMLS
semantic network.
First of all is that possible?  
If yes,
what is the involvement?  My plan is:
1)  To proceed with the implementation locally and
then check it in to be validated and deployed later (if it’s approved).  
2)  How ever I’m not very familiar with the subject
matter on how UMLS semantic relations are implemented in cTAKES. Is the << 
affects>>
relation will follow the same logic than LocationOfRelation  and 
DegreeOfLocation?  Which kind of IdentifiedAnnotation that the <<
affects>> relation will relates?  It
would be very appreciated if some one guides me on this matter.
 
Thanks Samir

Re: Common Type System across systems?

[samir chabou]

Just a thought ...
probably have some kind of interface (method) that allow to get the run time 
attributes of a certain typesystem. These method will hide some code 
complexity. For example BaseToken.getSuntenceNumber () will return the sentence 
number based on the selectCovering or indexCovering or isCovered 
Thanks
Samir





 From: Richard Eckart de Castilho 
To: dev@ctakes.apache.org 
Sent: Wednesday, October 2, 2013 12:07:36 PM
Subject: Re: Common Type System across systems?
 

Thanks for the reference, I'll have a look at it.

I don't plan to invent the ultimate type system :P Of course that would be
doomed to fail. I also don't plan to venture into the design of the special
medical types that cTAKES needs in addition.

I plan to make suggestions for the basic analysis levels (e.g. sentence,
token) and possibly work up from there into some of the lower linguistic 
analysis levels, as well as to suggest general design patterns. There are
also some ideas how to handle adoption to reduce changes to code to a minimum.

I think there is some realistic potential. But let's see how far this can be
pushed… if anywhere at all :) Maybe I'm too optimistic :P

-- Richard

On 02.10.2013, at 17:53, "Wu, Stephen T., Ph.D."  wrote:

> Richard, it'd be great if you are able to put dedicated effort to it,
> i.e., take the lead for (1) below!
> 
> Unfortunately, in our experience, you still need a lot of people and their
> time to be involved in (2), which often requires funding, and as mentioned
> in (2a) if it is not binding then people will be unlikely to adopt.  Maybe
> I'm overly pessimistic?
> 
> One specific portion of the cTAKES type system is that we make separate
> types for the clinical semantic groups.  The referential semantics portion
> of the type system was the main focus of our efforts (see reference below)
> due to its importance in the medical domain.  This is quite different than
> semantic structures, e.g., Discourse Representation Theory.  Richard, I'm
> interested in how you'd view the differences as someone who wasn't
> involved in their creation.
> 
> I think we made plenty of mistakes that make life difficult for people at
> a practical level, since we were designing it not necessarily even tied to
> UIMA.  But hopefully with your additional work it will be really good!
> 
> Anyways good luck! =P
> 
> stephen
> 
> * Wu, Stephen T, Vinod C Kaggal, Dmitriy Dligach, James J Masanz, Pei
> Chen, Lee Becker, Wendy W Chapman, Guergana K Savova, Hongfang Liu,
> Christopher G Chute. A common type system for clinical natural language
> processing . J Biomed Sem. 4:1.
> 2013.
> 
> On 10/1/13 2:53 PM, "Karthik Sarma"  wrote:
> 
>> This seems like a *very* challenging and involved problem to me...
>> 
>> On Tuesday, October 1, 2013, Pei Chen wrote:
>> 
>>> Agreed.
>>> Yes, I think this is slight augmentation and extension of the original
>>> vision of the clinical common type system- by having it work with other
>>> UIMA based NLP system.  Having worked on item (3) for cTAKES, I actually
>>> think the tough part will be getting consensus and agreement on a system
>>> between all parties and less on the required code changes.  Hence, just
>>> wanted to ping the community to gauge interest and see if this actually
>>> makes sense [It would be nice to plug in different POSTaggers or example
>>> without having to remap types].
>>> If we have a willing volunteer (Richard :)?) to perform some of the
>>> prelim
>>> analysis Q1 2014 with our existing type system, perhaps we can actually
>>> make this happen.
>>> 
>>> 4a) I think the SHARP4 development group has essentially moved to the
>>> cTAKES ASF community which is probably even better since it already has
>>> a
>>> meritocratic/governance mechanism to handle changes.
>>> 
>>> 
>>> 
>>> On Tue, Oct 1, 2013 at 10:39 AM, Wu, Stephen T., Ph.D.
>>> >wrote:
>>> 
 Pei et al,
 That was the vision for the SHARP "common type system", except it was
 meant to include medical-related projects rather than general
>>> projects.
 
 Steve's process below is probably the most realistic way to do things,
>>> and
 it's basically how we did the current cTAKES type system.
>>> Unfortunately,
 the "someone" doing #1 was me, and I didn't realize that it would be
>>> quite
 difficult.  I guess I know more about how to do it now but #1 and #2
>>> were
 surprisingly harder than I expected.  I'm adding a #4:
 
 (1) Have someone inspect the various type systems closely and make a
 proposal
  A. Know each of the type systems on their own.  Essential to
>>> visualize
 them appropriately, but it is still difficult to understand the
 implications of type changes just by looking. (By the way, we never
>>> came
 up with a really great automatic visualization tool, closest was a
>>> Protégé
 plugin. Excellent visualization would go a long way, especially

Re: sentence number in WordToken

[samir chabou]

thanks for the feed back it's a good point,
I did it also with selectCovering but as Richard mention I'll changed to 
indexCovering since it's faster.
Samir





 From: "Chen, Pei" 
To: "dev@ctakes.apache.org" ; samir chabou 
 
Sent: Monday, September 30, 2013 12:10:45 PM
Subject: RE: sentence number in  WordToken
 

Samir,
I think Richard has a good point here.   What is the use to require adding 
sentenceNumber() to BaseToken in the TypeSystem?
If it's only temporary, It may be a good idea to do it programmatically with 
local variable rather than modifying the type system and having it stored in 
the CAS...?

Maybe something like:
boolean a = JCasUtil.isCovered(JCas, BaseToken1, Sentence.class);
Boolean b = JCasUtil.isCovered(JCas, BaseToken2, Sentence.class);
--Pei


> -Original Message-
> From: Richard Eckart de Castilho [mailto:r...@apache.org]
> Sent: Monday, September 30, 2013 11:59 AM
> To: dev@ctakes.apache.org; samir chabou
> Subject: Re: sentence number in WordToken
> 
> Hi,
> 
> if you do many selectCovering calls, you may be faster using indexCovering
> once and then using the lookup index it produces.
> 
> IMHO type systems should not contain information that can easily be
> calculated at runtime (e.g. sentence number, token number, etc.).
> 
> Mind, I have no say here ;) Just my personal opinion.
> 
> -- Richard
> 
> On 30.09.2013, at 16:17, samir chabou  wrote:
> 
> > Hi Pei,
> >
> > I though
> > this may be have some use ...
> >
> > Because I
> > need to know if two or more words tokens belong to the same sentence;
> > and since WordToken does not define the feature sentence number. I
> > added it to the TypeSystem. These are the steps:
> >
> > 1)      I added the sentence number
> > features for the type BaseToken in TypeSystem.xml file (I choose the
> > supper class in order that the feature be propagated to all subclasses
> > (wordToken,SymboleToken,NumToken ...)
> >
> > 2)      In ctakes-core I in TokenizerAnnotatorPTB.java (methode
> annotateRange) I set the new feature
> > (BaseToken.sentenceNumber = sentence.getSentenceNumber()) as
> shown below :
> >
> > bta.setSentenceNumber(sentence.getSentenceNumber());
> >       bta.addToIndexes();
> >
> > 3)      Generate the JCASGen in the tab de TypeSystem of the
> > aggregate
> >
> > 4)      Add the feature in the source
> > tab of the aggregate
> >
> > Probably I
> > could have used as alternative:
> > List list = JCasUtil.selectCovering(aJcas, Sentence.class,
> > entity1.getBegin(), entity1.getEnd()); the issue with this is : if I
> > have many entities to be checked at the same time or if the entity1 is
> > found in many places, I have to add some if conditions to get sentence
> > number
> >
> >
> > Thanks
> > Samir

sentence number in WordToken

[samir chabou]

Hi Pei,

I though
this may be have some use …
 
Because I
need to know if two or more words tokens belong to the same sentence; and
since WordToken does not define the feature sentence number. I added it to the
TypeSystem. These are the steps:
 
1)  I added the sentence number
features for the type BaseToken in TypeSystem.xml file (I choose the supper
class in order that the feature be propagated to all subclasses
(wordToken,SymboleToken,NumToken …)
 
2)  In ctakes-core I in TokenizerAnnotatorPTB.java (methode annotateRange) 
I set the new feature
(BaseToken.sentenceNumber = sentence.getSentenceNumber()) as shown below :
 
bta.setSentenceNumber(sentence.getSentenceNumber());
  bta.addToIndexes();
 
3)  Generate the JCASGen in the tab de TypeSystem of the
aggregate
 
4)  Add the feature in the source
tab of the aggregate
 
Probably I
could have used as alternative:
List list = JCasUtil.selectCovering(aJcas, Sentence.class,
entity1.getBegin(), entity1.getEnd()); the issue with this is : if I have many
entities to be checked at the same time or if the entity1 is found in many
places, I have to add some if conditions to get sentence number 


Thanks
Samir

Re: examples of relation extractor annotations

[samir chabou]

Hi Dima,
Just to let you know that I'm now able to iterate on the relations. I wrote the 
cas as xmi using the CPE and after i read it. The relations are in the xmi cas 
even though they don't appear on the CVD.
Thanks for your help
Samir





 From: Dmitriy Dligach 
To: dev@ctakes.apache.org; samir chabou ; 
"u...@ctakes.apache.org"  
Sent: Tuesday, September 24, 2013 4:43:15 PM
Subject: Re: examples of relation extractor annotations
 

Samir,

Actually, the relations are represented in the type system differently. 
Please take a look at the BinaryTextRelation type. If you'd like an 
example of how to iterate over the relations, take a look at 
ctakes-relation-extractor/src/main/java/org/apache/ctakes/relationextractor/pipelines/RelationExtractorConsumer.java.
 
Once you get a hold of each BinaryTextRelation, you can determine the 
relation type by looking at the 'category' attribute.

Dima

On 09/23/2013 06:43 PM, samir chabou wrote:
> Hi Dima,
>
> I run RelationExtractorPipelineSingleCas.java and i got
> degree_of(severe pain/3, severe/2)
> degree_of(pain/3, severe/2)
> location_of(pain/3, wrists/6)
>
> but when i run UIMA_CVD--relation-extractor.launch - or 
> UIMA_CPE--relation-extractor.launch  on the hurd coded text  "He still is not 
> able to work because of severe pain involving his wrists." there is no 
> annotation of type:
> org.apache.ctakes.typesystem.type.relation.LocationOf
> org.apache.ctakes.typesystem.type.relation.DegreeOf
>
>
> I need to identify the relations  by their annotation type so i can iterate 
> on them
>
> Thanks
> Samir
>
>
>
>
> 
>   From: Dmitriy Dligach 
> To: dev@ctakes.apache.org; samir chabou ; 
> "u...@ctakes.apache.org" 
> Sent: Monday, September 23, 2013 4:38:35 PM
> Subject: Re: examples of relation extractor annotations
>  
>
> Samir,
>
> Could you please try to run 
> ctakes-relation-extractor/src/main/java/org/apache/ctakes/relationextractor/pipelines/RelationExtractorPipelineSingleCas.java
>  and see if it finds any relations in the hard-coded sample sentence. In that 
> sentence
>
> "He still is not able to work because of severe pain involving his wrists."
>
> you should see the following relations:
>
> degree_of(severe pain/3, severe/2)
> degree_of(pain/3, severe/2)
> location_of(pain/3, wrists/6)
>
> which will be printed to the console. The numbers next to each argument are 
> the UMLS semantic types (e.g. 6 = anatomical site).
>
> Dima
>
> On 09/21/2013 11:00 AM, samir chabou wrote:
>> Hi,
>> I tried to get examples of relation extractor annotations using 
>> RelationExtractorAggregate.xml. I tried the RelationExtractorAggregate on 
>> many input medical text but there was no annotation for any kind of 
>> relation. I even tried the bellow text used in 
>> RelationExtractorAnnotatorsTest but again no relation is annotated. Do you 
>> have idea or am i missing some thing ?
>>          "He had a slight fracture in the proximal right fibula.",
>>            "He had a slight fracture in the proximal right fibula .",
>>            "PRP VBD DT JJ NN IN DT JJ JJ NN .");
>>
>>
>> Thanks
>> Samir
>>
> -- Dmitriy Dligach, PhD
> Research Fellow
> Children's Hospital Informatics Program
> Boston Children's Hospital and Harvard Medical School
> (617) 919-3596

-- 
Dmitriy Dligach, PhD
Research Fellow
Children's Hospital Informatics Program
Boston Children's Hospital and Harvard Medical School
(617) 919-3596

Re: UMLS verification - host unreachable

[samir chabou]

yes.  I encountered the same since this morning 10 oclock
Samir





 From: Kannan Thiagarajan 
To: dev@ctakes.apache.org 
Sent: Wednesday, September 25, 2013 12:40:06 PM
Subject: UMLS verification - host unreachable
 

Hello,

I get a host unreachable error with the following URL during UMLS license
verification. This has been working fine so far but I have noticed similar
issue some time ago.

uts-ws.nlm.nih.gov

I understand this is outside of ctakes itself, but just wanted to see if
anyone else has encountered this and if there is a workaround.

Thanks

Re: examples of relation extractor annotations

[samir chabou]

Thanks Dima for these extra info. that may help me
I'll try it and let you know.
Samir





 From: Dmitriy Dligach 
To: dev@ctakes.apache.org; samir chabou ; 
"u...@ctakes.apache.org"  
Sent: Tuesday, September 24, 2013 4:43:15 PM
Subject: Re: examples of relation extractor annotations
 

Samir,

Actually, the relations are represented in the type system differently. 
Please take a look at the BinaryTextRelation type. If you'd like an 
example of how to iterate over the relations, take a look at 
ctakes-relation-extractor/src/main/java/org/apache/ctakes/relationextractor/pipelines/RelationExtractorConsumer.java.
 
Once you get a hold of each BinaryTextRelation, you can determine the 
relation type by looking at the 'category' attribute.

Dima

On 09/23/2013 06:43 PM, samir chabou wrote:
> Hi Dima,
>
> I run RelationExtractorPipelineSingleCas.java and i got
> degree_of(severe pain/3, severe/2)
> degree_of(pain/3, severe/2)
> location_of(pain/3, wrists/6)
>
> but when i run UIMA_CVD--relation-extractor.launch - or 
> UIMA_CPE--relation-extractor.launch  on the hurd coded text  "He still is not 
> able to work because of severe pain involving his wrists." there is no 
> annotation of type:
> org.apache.ctakes.typesystem.type.relation.LocationOf
> org.apache.ctakes.typesystem.type.relation.DegreeOf
>
>
> I need to identify the relations  by their annotation type so i can iterate 
> on them
>
> Thanks
> Samir
>
>
>
>
> 
>   From: Dmitriy Dligach 
> To: dev@ctakes.apache.org; samir chabou ; 
> "u...@ctakes.apache.org" 
> Sent: Monday, September 23, 2013 4:38:35 PM
> Subject: Re: examples of relation extractor annotations
>  
>
> Samir,
>
> Could you please try to run 
> ctakes-relation-extractor/src/main/java/org/apache/ctakes/relationextractor/pipelines/RelationExtractorPipelineSingleCas.java
>  and see if it finds any relations in the hard-coded sample sentence. In that 
> sentence
>
> "He still is not able to work because of severe pain involving his wrists."
>
> you should see the following relations:
>
> degree_of(severe pain/3, severe/2)
> degree_of(pain/3, severe/2)
> location_of(pain/3, wrists/6)
>
> which will be printed to the console. The numbers next to each argument are 
> the UMLS semantic types (e.g. 6 = anatomical site).
>
> Dima
>
> On 09/21/2013 11:00 AM, samir chabou wrote:
>> Hi,
>> I tried to get examples of relation extractor annotations using 
>> RelationExtractorAggregate.xml. I tried the RelationExtractorAggregate on 
>> many input medical text but there was no annotation for any kind of 
>> relation. I even tried the bellow text used in 
>> RelationExtractorAnnotatorsTest but again no relation is annotated. Do you 
>> have idea or am i missing some thing ?
>>          "He had a slight fracture in the proximal right fibula.",
>>            "He had a slight fracture in the proximal right fibula .",
>>            "PRP VBD DT JJ NN IN DT JJ JJ NN .");
>>
>>
>> Thanks
>> Samir
>>
> -- Dmitriy Dligach, PhD
> Research Fellow
> Children's Hospital Informatics Program
> Boston Children's Hospital and Harvard Medical School
> (617) 919-3596

-- 
Dmitriy Dligach, PhD
Research Fellow
Children's Hospital Informatics Program
Boston Children's Hospital and Harvard Medical School
(617) 919-3596

Re: examples of relation extractor annotations

[samir chabou]

Hi Dima,

I run RelationExtractorPipelineSingleCas.java and i got 
degree_of(severe pain/3, severe/2)
degree_of(pain/3, severe/2)
location_of(pain/3, wrists/6)

but when i run UIMA_CVD--relation-extractor.launch - or 
UIMA_CPE--relation-extractor.launch  on the hurd coded text  "He still is not 
able to work because of severe pain involving his wrists." there is no 
annotation of type: 
org.apache.ctakes.typesystem.type.relation.LocationOf
org.apache.ctakes.typesystem.type.relation.DegreeOf


I need to identify the relations  by their annotation type so i can iterate on 
them

Thanks
Samir





 From: Dmitriy Dligach 
To: dev@ctakes.apache.org; samir chabou ; 
"u...@ctakes.apache.org"  
Sent: Monday, September 23, 2013 4:38:35 PM
Subject: Re: examples of relation extractor annotations
 

Samir,

Could you please try to run 
ctakes-relation-extractor/src/main/java/org/apache/ctakes/relationextractor/pipelines/RelationExtractorPipelineSingleCas.java
 and see if it finds any relations in the hard-coded sample sentence. In that 
sentence

"He still is not able to work because of severe pain involving his wrists."

you should see the following relations:

degree_of(severe pain/3, severe/2)
degree_of(pain/3, severe/2)
location_of(pain/3, wrists/6)

which will be printed to the console. The numbers next to each argument are the 
UMLS semantic types (e.g. 6 = anatomical site).

Dima

On 09/21/2013 11:00 AM, samir chabou wrote:
> Hi,
> I tried to get examples of relation extractor annotations using 
> RelationExtractorAggregate.xml. I tried the RelationExtractorAggregate on 
> many input medical text but there was no annotation for any kind of relation. 
> I even tried the bellow text used in RelationExtractorAnnotatorsTest but 
> again no relation is annotated. Do you have idea or am i missing some thing ?
>        "He had a slight fracture in the proximal right fibula.",
>          "He had a slight fracture in the proximal right fibula .",
>          "PRP VBD DT JJ NN IN DT JJ JJ NN .");
> 
> 
> Thanks
> Samir
> 

-- Dmitriy Dligach, PhD
Research Fellow
Children's Hospital Informatics Program
Boston Children's Hospital and Harvard Medical School
(617) 919-3596

examples of relation extractor annotations

[samir chabou]

Hi,
I tried to get examples of relation extractor annotations using 
RelationExtractorAggregate.xml. I tried the RelationExtractorAggregate on many 
input medical text but there was no annotation for any kind of relation. I even 
tried the bellow text used in RelationExtractorAnnotatorsTest but again no 
relation is annotated. Do you have idea or am i missing some thing ?
  "He had a slight fracture in the proximal right fibula.",
    "He had a slight fracture in the proximal right fibula .",
    "PRP VBD DT JJ NN IN DT JJ JJ NN .");


Thanks
Samir

Re: Apache cTAKES > cTAKES 3.1 User Install Guide

[samir chabou]

Hi James

When you say "Non-English environments" is using  cTAKES  in a french Operating 
System (windows 7) is a case ?  
Samir




 From: "Masanz, James J." 
To: "'dev@ctakes.apache.org'" ; 
"'u...@ctakes.apache.org'"  
Sent: Tuesday, September 17, 2013 5:46:36 PM
Subject: FW: Apache cTAKES > cTAKES 3.1 User Install Guide
 

The comment "It would be great if someone could say something about Non-English 
environments." was made on the 3.1 User Install Guide page.

If anyone has used cTAKES in a non-English environment and would share any 
information here, I'd be happy to help incorporate the information into the 
documentation.

-- James

From: masanz.james+caf_=masanz.james=mayo@gmail.com 
[mailto:masanz.james+caf_=masanz.james=mayo@gmail.com] On Behalf Of 
Anonymous (Confluence)
Sent: Tuesday, September 17, 2013 2:59 AM
To: masanz.ja...@gmail.com
Subject: [CONF] Apache cTAKES > cTAKES 3.1 User Install Guide

cTAKES 3.1 User Install Guide
Page comment added by Anonym 

It would be great if someone could say something about Non-English environments.
Stop watching page | Change email notification preferences 
View Online | Reply To This 

bugs to correct in release v3.1

[samir chabou]

the the print screen of the error details that i included in the e-mail didn't 
appear.

so i resend you the e-mail:


Hi James,
the UIMA_CVD--relation-extractor.launch under project ctakes-relation-extractor 
rise errors:

A) the error is: Launching UIMA_CVD--relation-extractor has encountered a 
problem 
<>


B) to correct this :
1) I edited UIMA_CVD--relation-extractor.launch
2) I changed the value attribute in 
from value="relation-extractor" to value="ctakes-relation-extractor"

Thanks 

Samir


 


________
 From: samir chabou 
To: "dev@ctakes.apache.org"  
Sent: Friday, September 13, 2013 5:55:11 AM
Subject: bugs to correct in release v3.1
 




Hi James,
the UIMA_CVD--relation-extractor.launch under project ctakes-relation-extractor 
rise errors:


to correct this :
1) I edited UIMA_CVD--relation-extractor.launch
2) I changed the value attribute in 
from value="relation-extractor" to value="ctakes-relation-extractor"

Thanks 

Samir



 From: "Chen, Pei" 
To: "dev@ctakes.apache.org"  
Sent: Thursday, September 12, 2013 10:49:39 PM
Subject: RE: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)
 

Hi Sarma,
Yes, we're able to commit to the repo's now (no more size limit issue):
https://dist.apache.org/repos/dist/dev/ctakes/
https://dist.apache.org/repos/dist/release/ctakes/

Still waiting for the svnpubsub setup so that it will automatically update the 
mirrors. (https://issues.apache.org/jira/browse/INFRA-6731)
Once that's done, I think we should be able to update the downloads page...

--Pei

> -Original Message-
> From: ksa...@gmail.com [mailto:ksa...@gmail.com] On Behalf Of Karthik
> Sarma
> Sent: Thursday, September 12, 2013 1:04 PM
> To: dev@ctakes.apache.org
> Subject: Re: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)
> 
> Has there been any movement on these issues? Just took a look at the site
> and it looks like 3.1 isn't up yet, but haven't really been following this
> thread...
> 
> 
> 
> 
> 
> --
> Karthik Sarma
> UCLA Medical Scientist Training Program Class of 20??
> Member, UCLA Medical Imaging & Informatics Lab Member, CA Delegation
> to the House of Delegates of the American Medical Association
> ksa...@ksarma.com
> gchat: ksa...@gmail.com
> linkedin: www.linkedin.com/in/ksarma
> 
> 
> On Tue, Sep 10, 2013 at 7:42 AM, Chen, Pei
> wrote:
> 
> > Hi James,
> > I just ping'd infra- working on this- stay tuned...
> >
> > --Pei
> >
> > > -Original Message-
> > > From: Masanz, James J. [mailto:masanz.ja...@mayo.edu]
> > > Sent: Tuesday, September 10, 2013 7:03 AM
> > > To: dev@ctakes.apache.org
> > > Subject: RE: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)
> > >
> >
 >
> > > Pei, is there something you can do as VP to push action on
> > > INFRA-6731 and/or INFRA-6728? Neither has had activity by Infra
> > > since 9/6 (nor has
> > > INFRA-6721)
> > >
> > > There was a typo in previous email I sent, meant INFRA-6731, not
> > > INFRA- 6721.
> > >
> > > INFRA-6731 "please create http://www.apache.org/dist/ctakes";
> > >
> > > www.apache.org/dist/ctakes still does not exist so I still can't
> > > even
> > try to
> > > manually push the zips and tars for 3.1.0
> > >
> > >
> > > [1] https://issues.apache.org/jira/browse/INFRA-6731
> > > [2] https://issues.apache.org/jira/browse/INFRA-6728
> > >
> > > 
> > > From: Masanz, James J.
> > > Sent: Monday, September 09, 2013 1:29 PM
> > > To: 'dev@ctakes.apache.org'
> > > Subject: RE: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)
> > >
> > > Fyi, cTAKES 3.1 is not released yet, but hopefully soon. Although I
> > don't see
> > > any activity by infra on either of these [1] [2], I do see that the
> > following
> > > directory did end up created eventually (perhaps by the svnpubsub
> > > process?) and populated with the smaller files so I will push the
> > > zips
> > and tars
>
 > > there manually (sftp)  provided I have the authority to do that.
> > >
> > > [1] https://issues.apache.org/jira/browse/INFRA-6721
> > > [2] https://issues.apache.org/jira/browse/INFRA-6728
> > >
> > > INFRA-6728      unable to commit large files to
> > > dist.apache.org/repos/dist/release/ctakes
> > > INFRA-6721      please create dir for
> > > https:

bugs to correct in release v3.1

[samir chabou]

Hi James,
the UIMA_CVD--relation-extractor.launch under project ctakes-relation-extractor 
rise errors:


to correct this :
1) I edited UIMA_CVD--relation-extractor.launch
2) I changed the value attribute in 
from value="relation-extractor" to value="ctakes-relation-extractor"

Thanks 

Samir



 From: "Chen, Pei" 
To: "dev@ctakes.apache.org"  
Sent: Thursday, September 12, 2013 10:49:39 PM
Subject: RE: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)
 

Hi Sarma,
Yes, we're able to commit to the repo's now (no more size limit issue):
https://dist.apache.org/repos/dist/dev/ctakes/
https://dist.apache.org/repos/dist/release/ctakes/

Still waiting for the svnpubsub setup so that it will automatically update the 
mirrors. (https://issues.apache.org/jira/browse/INFRA-6731)
Once that's done, I think we should be able to update the downloads page...

--Pei

> -Original Message-
> From: ksa...@gmail.com [mailto:ksa...@gmail.com] On Behalf Of Karthik
> Sarma
> Sent: Thursday, September 12, 2013 1:04 PM
> To: dev@ctakes.apache.org
> Subject: Re: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)
> 
> Has there been any movement on these issues? Just took a look at the site
> and it looks like 3.1 isn't up yet, but haven't really been following this
> thread...
> 
> 
> 
> 
> 
> --
> Karthik Sarma
> UCLA Medical Scientist Training Program Class of 20??
> Member, UCLA Medical Imaging & Informatics Lab Member, CA Delegation
> to the House of Delegates of the American Medical Association
> ksa...@ksarma.com
> gchat: ksa...@gmail.com
> linkedin: www.linkedin.com/in/ksarma
> 
> 
> On Tue, Sep 10, 2013 at 7:42 AM, Chen, Pei
> wrote:
> 
> > Hi James,
> > I just ping'd infra- working on this- stay tuned...
> >
> > --Pei
> >
> > > -Original Message-
> > > From: Masanz, James J. [mailto:masanz.ja...@mayo.edu]
> > > Sent: Tuesday, September 10, 2013 7:03 AM
> > > To: dev@ctakes.apache.org
> > > Subject: RE: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)
> > >
> > >
> > > Pei, is there something you can do as VP to push action on
> > > INFRA-6731 and/or INFRA-6728? Neither has had activity by Infra
> > > since 9/6 (nor has
> > > INFRA-6721)
> > >
> > > There was a typo in previous email I sent, meant INFRA-6731, not
> > > INFRA- 6721.
> > >
> > > INFRA-6731 "please create http://www.apache.org/dist/ctakes";
> > >
> > > www.apache.org/dist/ctakes still does not exist so I still can't
> > > even
> > try to
> > > manually push the zips and tars for 3.1.0
> > >
> > >
> > > [1] https://issues.apache.org/jira/browse/INFRA-6731
> > > [2] https://issues.apache.org/jira/browse/INFRA-6728
> > >
> > > 
> > > From: Masanz, James J.
> > > Sent: Monday, September 09, 2013 1:29 PM
> > > To: 'dev@ctakes.apache.org'
> > > Subject: RE: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)
> > >
> > > Fyi, cTAKES 3.1 is not released yet, but hopefully soon. Although I
> > don't see
> > > any activity by infra on either of these [1] [2], I do see that the
> > following
> > > directory did end up created eventually (perhaps by the svnpubsub
> > > process?) and populated with the smaller files so I will push the
> > > zips
> > and tars
> > > there manually (sftp)  provided I have the authority to do that.
> > >
> > > [1] https://issues.apache.org/jira/browse/INFRA-6721
> > > [2] https://issues.apache.org/jira/browse/INFRA-6728
> > >
> > > INFRA-6728      unable to commit large files to
> > > dist.apache.org/repos/dist/release/ctakes
> > > INFRA-6721      please create dir for
> > > https://dist.apache.org/repos/dist/release/ctakes
> > >
> > > -- James
> > >
> > > -Original Message-
> > > From: Masanz, James J.
> > > Sent: Wednesday, September 04, 2013 10:36 PM
> > > To: 'dev@ctakes.apache.org'
> > > Subject: RE: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)
> > >
> > >
> > > The nexus staging repository has been released in the meantime.
> > >
> > > 
> > > From: Masanz, James J.
> > > Sent: Wednesday, September 04, 2013 3:22 PM
> > > To: 'dev@ctakes.apache.org'
> > > Subject: RE: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)
> > >
> > > FYI, awaiting further action on
> > > https://issues.apache.org/jira/browse/INFRA-6721
> > > before I can publish the release - getting an error about the size
> > > of
> > the files
> > > even though each is < 1 GB.
> > >
> > > -- James
> > >
> > > -Original Message-
> > > From: dev-return-1957-Masanz.James=mayo@ctakes.apache.org
> > > [mailto:dev-return-1957-Masanz.James=mayo@ctakes.apache.org]
> On
> > > Behalf Of Masanz, James J.
> > > Sent: Wednesday, September 04, 2013 9:40 AM
> > > To: 'dev@ctakes.apache.org'
> > > Subject: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)
> > >
> > >
> > > More than 72 hours has passed. The vote for Apache cTAKES 3.1.0
> *passes*
> > > [1] with three  +1 votes   (3 binding)
> > >
> > > +1 (binding)
> > > Pei Chen
> > > Britt Fitch
>

Re: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)

[samir chabou]

Hi James,
These is a minors bugs that i have corrected, it may be of use for the next 
release:
I- the XCasWriterCasConsumer.xml analysis engine (cas-consumer of ctakes-core) 
referes to org.apache.uima.examples.cpe.XCasWriterCasConsumer which does not 
exists.
1) I found the java class XCasWriterCasConsume.java from an old version of 
cTakes and copied in org.apache.ctakes.core.cc   
2) Changed org.apache.uima.examples.cpe.XCasWriterCasConsumer by 
org.apache.ctakes.core.cc.XCasWriterCasConsumer 

thanks
Samir



 From: "Masanz, James J." 
To: "'dev@ctakes.apache.org'"  
Sent: Monday, September 9, 2013 7:29:47 PM
Subject: RE: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)
 

Fyi, cTAKES 3.1 is not released yet, but hopefully soon. Although I don't see 
any activity by infra on either of these [1] [2], I do see that the following 
directory did end up created eventually (perhaps by the svnpubsub process?) and 
populated with the smaller files so I will push the zips and tars there 
manually (sftp)  provided I have the authority to do that.

[1] https://issues.apache.org/jira/browse/INFRA-6721
[2] https://issues.apache.org/jira/browse/INFRA-6728

INFRA-6728    unable to commit large files to 
dist.apache.org/repos/dist/release/ctakes
INFRA-6721    please create dir for 
https://dist.apache.org/repos/dist/release/ctakes

-- James

-Original Message-
From: Masanz, James J. 
Sent: Wednesday, September 04, 2013 10:36 PM
To: 'dev@ctakes.apache.org'
Subject: RE: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)


The nexus staging repository has been released in the meantime.


From: Masanz, James J.
Sent: Wednesday, September 04, 2013 3:22 PM
To: 'dev@ctakes.apache.org'
Subject: RE: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)

FYI, awaiting further action on
https://issues.apache.org/jira/browse/INFRA-6721
before I can publish the release - getting an error about the size of the files 
even though each is < 1 GB.

-- James

-Original Message-
From: dev-return-1957-Masanz.James=mayo@ctakes.apache.org 
[mailto:dev-return-1957-Masanz.James=mayo@ctakes.apache.org] On Behalf Of 
Masanz, James J.
Sent: Wednesday, September 04, 2013 9:40 AM
To: 'dev@ctakes.apache.org'
Subject: [RESULT] [VOTE] Release Apache cTAKES 3.1 (rc3)


More than 72 hours has passed. The vote for Apache cTAKES 3.1.0 *passes* [1] 
with three  +1 votes   (3 binding)

+1 (binding)
Pei Chen
Britt Fitch
James Masanz

There were no -1 or +0 votes cast.

Thanks to Tim M for the testing he did.
And thanks to all who looked at previous release candidates for 3.1.0.

I will be publishing the release [2], then will announce the release as soon as 
artifacts will be available.

Thanks to everyone for participating!

[1] http://markmail.org/message/i4ovjd35legofibk
[2] 
http://ctakes.apache.org/ctakes-release-guide.html#publishing-artifacts-to-dist

Regards,
James Masanz

-Original Message-
From: dev-return-1949-Masanz.James=mayo@ctakes.apache.org 
[mailto:dev-return-1949-Masanz.James=mayo@ctakes.apache.org] On Behalf Of 
Britt Fitch
Sent: Tuesday, September 03, 2013 8:15 PM
To: dev@ctakes.apache.org
Subject: Re: [VOTE] Release Apache cTAKES 3.1 (rc3)

+1 here as well.


On Tue, Sep 3, 2013 at 11:47 AM, Pei Chen  wrote:

> James,
> +1
> [x] +1 Release the packages as Apache cTAKES 3.1.0
>
> Below is what I verified so far (plan to go through the documentation in
> parallel):
>
> Tested:
> 1- Downloaded src and bin tz
> 2- Verified Signatures
> 3- Unpacked and able to compile from source (mvn compile automatically
> downloads umls resources)
> 3a - Able to run AggregatePlainTextProcessor using CVD from $mvn -PrunCVD
> 3b - Able to run a test note using AggregatePlainTextUMLSProcessor
> 4- Unpacked bin
> 4a - Download and merge resources folder from
>
> http://sourceforge.net/projects/ctakesresources/files/ctakes-resources-3.1.0.zip/download
> 4b - Added umls credentials to java args
> 4c - Able to run a sample note usning AggregatePlainTextUMLSProcessor
>
>
> Notes:
> 1- User will get a misleading SQLException due to I/O umls file not found
> if they do not download the umls resources.  I don't think it's a show
> stopper.
> 2- For consistency, we should copy over the NOTICE/LICENSE for the new
> template-filler module.  I don't think it's a show stopper, everything
> there is included in the root NOTICE/LICENSE already.
> 3- runctakesCVD.sh warning about log4j.xml not found.  Should we just
> remove it from the sh if it's not there?
> 4- got a GC OOM heap exceed error with the sample- let's increase the mem
> requirements to 2g in the next release due to more models loaded in the
> newer modules?
>
> $ gpg --verify apache-ctakes-3.1.0-bin.tar.gz.asc
> gpg: Signature made Fri Aug 30 10:18:26 2013 EDT using RSA key ID FD239395
> gpg: Good signature from "James Joseph Masanz (CODE SIGNING KEY) <
> james-mas...@apache.org>"
> gpg: WARNING: This key i

Re: Build error on ctakes-3.1_RC2

[samir chabou]

hi,
this is may be useful for the next release. SideEffect SideEffectAnnotator.xml 
still refering to cTAKES2.5 files ../type_system/SideEffectTypeSystem.xml and 
Commun type. I corrected the error in SideEffectAnnotator.xml  in stead of :

 
  
    
  
      I put :

  
  
    
  
    thanks
Samir


 From: Kim Ebert 
To: dev@ctakes.apache.org 
Cc: "Masanz, James J."  
Sent: Monday, August 26, 2013 12:32:41 PM
Subject: Re: Build error on ctakes-3.1_RC2
 

Thanks All.

I'm just trying to do some prep work to get ready for cTAKES 3.1, so
building is pretty important.

Thanks,

Kim Ebert
1.801.669.7342
Perfect Search Corp
http://www.perfectsearchcorp.com/

On 08/26/2013 08:12 AM, Masanz, James J. wrote:
> thanks! I was looking at  rather than 
> 
> From: dev-return-1881-Masanz.James=mayo@ctakes.apache.org 
> [dev-return-1881-Masanz.James=mayo@ctakes.apache.org] on behalf of Pei 
> Chen [chen...@apache.org]
> Sent: Monday, August 26, 2013 9:09 AM
> To: dev@ctakes.apache.org
> Subject: Re: Build error on ctakes-3.1_RC2
>
> Hi James,
> I meant the property here in the root pom.xml:
> 3.1.0
>
>
>
> On Mon, Aug 26, 2013 at 10:05 AM, Masanz, James J. 
> wrote:
>
>> Pei,
>>
>> I'm sorry, I don't understand your comment.
>> the root pom for tag ctakes-3.1.0-rc2 now has
>>
>> 3.1.0-SNAPSHOT
>>
>> 
>> From: 
>> dev-return-1878-Masanz.James=mayo@ctakes.apache.org[dev-return-1878-Masanz.James=
>> mayo@ctakes.apache.org] on behalf of Chen, Pei [
>> pei.c...@childrens.harvard.edu]
>> Sent: Sunday, August 25, 2013 4:00 PM
>> To: 
>> Cc: dev@ctakes.apache.org
>> Subject: Re: Build error on ctakes-3.1_RC2
>>
>> I think it should be 3.1.0-SNAPSHOT...?
>>
>> Sent from my iPad
>>
>> On Aug 25, 2013, at 2:39 PM, "Masanz, James J." 
>> wrote:
>>
>>> I deleted and re-created the tag from a revision that still had
>>>
>>> 3.1.0-SNAPSHOT
>>>
>>> Note rc2 has not been released for a vote yet because there appears to
>> be a problem with the tokenizer sometimes creating PunctionToken when it
>> should be creating ContractionToken, which I will look into on Monday.
>>> -- James
>>>
>>> -Original Message-
>>> From: dev-return-1875-Masanz.James=mayo@ctakes.apache.org [mailto:
>> dev-return-1875-Masanz.James=mayo@ctakes.apache.org] On Behalf Of Pei
>> Chen
>>> Sent: Friday, August 23, 2013 9:01 PM
>>> To: dev@ctakes.apache.org
>>> Subject: Re: Build error on ctakes-3.1_RC2
>>>
>>> James,
>>> I could recreate Kim's issue with the RC2 tag.  I believe the root
>> pom.xml
>>> should be; otherwise it won't build:
>>>  +3.1.0-SNAPSHOT instead of
>>> this:
>>> -3.1.0
>>> 
>>>
>>>
>>> On Fri, Aug 23, 2013 at 5:20 PM, Kim Ebert
>>> wrote:
>>>
 Hi All,

 I am trying to compile ctakes and I am getting a build error. Any ideas
 of what I might be doing wrong?

 [ERROR] Failed to execute goal on project ctakes-utils: Could not
 resolve dependencies for project
 org.apache.ctakes:ctakes-utils:jar:3.1.0-SNAPSHOT: Could not find
 artifact org.apache.ctakes:ctakes-type-system:jar:3.1.0 in central
 (http://repo.maven.apache.org/maven2) -> [Help 1]

 Thanks,

 --
 Kim Ebert
 1.801.669.7342
 Perfect Search Corp
 http://www.perfectsearchcorp.com/

Re: pico pipeline

[samir chabou]

Hi Pei, thanks for your feedback:
Yes there was some typo, I also  added some
cTakes annotators in the pico pipeline (see attached). 
 
Our purpose is: 
   1) To improve the
PICO recognition in abstracts text compared to what we are currently
testing with metamap
 
2 2) Through out my reading of some articles there
are some authors find that PICO is a useful organizing structure for clinical 
questions, otherthey suggest it is less suitable for DiagnosisorPrognosis. From 
the few pico’s cases that I saw (I
need to see a good sample of Pico’s Dignosis, Prognosis), I think :
 
a.   In the case of the Diagnosis and Prognosis the
focus of the question is most likely Output (O?)  (Need to be confirmed). Since 
the Output is usually tend to be the relation between the Problem and 
Intervention (I -- O -->P or C -- O --> P), consequently one of the thing to 
enhance Pico structure
for Prognosis and Dignosis is to enhance the aspect of relation recognition in
the text, to do so we are planning to write code/logic to add some relations
which will help the recognition of the O aspect.

b.   I also noticed that these
suggestions about pico on prognosis were advanced before 2006 where the focus
of the research was on the named entity recognition (NER) but few were done
about relation recognition.  I think
since then, the NLP techniques are evolved considerably  to be more efficient 
in relation recognition
which will help our purpose.
 
 
Samir




 From: "Chen, Pei" 
To: samir chabou ; "dev@ctakes.apache.org" 
 
Sent: Friday, August 23, 2013 10:41:46 AM
Subject: RE: pico pipeline
 

Hi Samir,
Perhaps others can chime in as well as I'm not too familiar with the proposed 
pipeline-
But it looks really interesting- especially the higher level components such as 
the Intervention Annotator (are you planning to write code/logic to 'infer' 
what the cause or prognosis was?,etc).
Yes, one can certainly use the already annotated data from the existing 
components.  Some notes:

-          SentenceDetector, Tokenizer is listed twice in the pipeline (is that 
just a typo?)

-          I think there may be other components that you may want to include 
which might help with your higher level annotators (some of which may or may 
not be available in mata-map):

o   Assertion (Negation, Subject, History-of, etc.)

o   Co Reference

o   Semantic Role Labeler

o   Temporal?
--Pei

From: samir chabou [mailto:samir...@yahoo.com]
Sent: Thursday, August 22, 2013 11:43 AM
To: Chen, Pei; dev@ctakes.apache.org
Subject: pico pipeline

Hi Pei,
I'm trying to use ctakes to annotate PICO question concepts. I attached to you 
the pipeline that I'll construct to do so. Please can you have a quick look to 
the attached file and tell me if i'm on the right track or if you have some 
suggestions.
Thanks a lot
Samir

pico pipeline

[samir chabou]

Hi Pei,
I'm trying to use ctakes to annotate PICO question concepts. I attached to you 
the pipeline that I'll construct to do so. Please can you have a quick look to 
the attached file and tell me if i'm on the right track or if you have some 
suggestions.
Thanks a lot
Samir

Re: umls lookup issue

[samir chabou]

Hi Pei,
I have a good news for you :) the issue was resolved. The problem was related 
to the missing umls2011ab which is not found in the resource folder. 

I think (i'm not sure) cTakes tries to apply the query against the resource 
umls2011ab when it does not find it it tries to apply it on the umls server 
side and that takes time.

This morning i added the umls2011ab resource and I'm able to run the test 
within 2 min.

Also over the week end,  i created a local db for the two tables umls_ms_2011ab 
and umls_snomed_map I load them with a sample of data. The performance on the 
local db is less than 1 min for the same test.

Thanks a lot for your support
Samir





 From: "Chen, Pei" 
To: "dev@ctakes.apache.org" ; samir chabou 
 
Sent: Monday, August 19, 2013 10:40:34 AM
Subject: RE: umls lookup issue 
 


 
Hi Samir,
I ran your attached DefaultAggregateUMLSPipleine and abstract text file (Using 
the trunk codebase and the -XX:+UseConcMarkSweepGC -Xms500M -Xmx1600M args) .  
It took about 2min16secs (see attached results output) to finish.
From the initial looks, it doesn’t appear to be a loop or a bug to me and seems 
more like a local setup/configuration issue…
 
Re: ctakes-resource-umls2011ab, did the system print out an error/exception 
message?  If it was missing, it should have thrown an exception and not hang 
for 3hrs but eventually finish.
--Pei
 
 
From:samir chabou [mailto:samir...@yahoo.com] 
Sent: Sunday, August 18, 2013 1:35 PM
To: dev@ctakes.apache.org
Subject: Re: umls lookup issue 
 
Hi Pei,
i'm sorry if i bothered you a bit with my umls lookup issue. I just noticed 
that I have an error in the pom of the lookup dictionary project, that may a be 
a clue to my problem. Can you please have a fast look to the attached file 
where i put the details of
 the error - it looks as if i'm missing a kind of project 
ctakes-resource-umls2011ab.
Thanks Samir
 

________
 
From:"Chen, Pei" 
To: samir chabou  
Cc: "dev@ctakes.apache.org"  
Sent: Thursday, August 15, 2013 10:23:50 PM
Subject: RE: umls lookup issue 

Hi Samir,
Do you have a sample sentence that causes the 3hr run?
Also could you attach the AggregatePipeline.xml configuration used? In case, 
someone else on the dev list may have encountered this in the past already.

I'll try and see if I can recreate it.
--Pei

From: samir chabou [samir...@yahoo.com]
Sent: Thursday, August 15, 2013 7:07 PM
To: Chen, Pei
Subject: Re: umls lookup issue

Hi Pei,
we did more debuging and it's the lookup call below (higlighted in yelleow) 
that causes the delay.

performLookup is in DictionaryLookupAnnotator.java

private void performLookup(JCas jcas, LookupSpec ls, List lookupTokenList,
                    Map ctxMap) throws Exception
      {
            // sort the lookup tokens
            Collections.sort(lookupTokenList, 
LookupTokenComparator.getInstance() );

            // perform lookup
            Collection lookupHitCol = null;

            LookupAlgorithm la = (LookupAlgorithm) ls.getLookupAlgorithm();
            lookupHitCol = la.lookup(lookupTokenList, ctxMap);

Samir



________
From: "Chen, Pei" 
To: "dev@ctakes.apache.org" 
Cc: samir chabou 
Sent: Thursday, August 15, 2013 9:00:37 AM
Subject: RE: umls lookup issue

Hi Samir,
[including the public dev list]
Thanks for opening up a new thread on this issue.
Would you be able to help narrow down the sentence that you believe is causing 
the NP2LookupWindow to take 3h to process?  I can’t seem to reproduce it on my 
end.
I vaguely remember someone running into something where it could go into a 
loop, so hopefully maybe they can also chime in…

--Pei

From: samir chabou [mailto:samir...@yahoo.com]
Sent: Wednesday, August 14, 2013 7:30 PM
To: Chen, Pei
Subject: Re: umls lookup issue


specifically the NP2LookupWindow that causes de delay

From: samir chabou 
To: "Chen, Pei" 
Sent: Wednesday, August 14, 2013 7:21:18 PM
Subject: Re: umls lookup issue


Hi Pei
I removed the LookupWindowAnnotator went very fast less than 1 min but there 
was no annotations for EntityMention and EventMention, it looks there is some 
thinh wrong with the LookupWindowAnnotator
Samir




From: samir chabou 
To: "Chen, Pei" 
Sent: Wednesday, August 14, 2013 7:11:57 PM
Subject: Re: umls lookup issue

Hi Pei
I removed the lookupwindowannotation went very fast less than 1 min but there 
was no annotations for EntityMention and EventMention, it looks there is some 
thinh wrong with the lookupwindowannotation
Samir



From: "Chen, Pei" 
To: samir chabou 
Sent: Wednesday, August 14, 2013 3:40:46 PM
Subject: RE: umls lookup issue

That is strange- it shouldn’t take that long.  I wonder if it’s goi

Re: umls lookup issue

[samir chabou]

Hi Pei,
i'm sorry if i bothered you a bit with my umls lookup issue. I just noticed 
that I have an error in the pom of the lookup dictionary project, that may a be 
a clue to my problem. Can you please have a fast look to the attached file 
where i put the details of the error - it looks as if i'm missing a kind of 
project ctakes-resource-umls2011ab.

Thanks Samir




 From: "Chen, Pei" 
To: samir chabou  
Cc: "dev@ctakes.apache.org"  
Sent: Thursday, August 15, 2013 10:23:50 PM
Subject: RE: umls lookup issue 
 

Hi Samir,
Do you have a sample sentence that causes the 3hr run?
Also could you attach the AggregatePipeline.xml configuration used? In case, 
someone else on the dev list may have encountered this in the past already.

I'll try and see if I can recreate it.
--Pei
____
From: samir chabou [samir...@yahoo.com]
Sent: Thursday, August 15, 2013 7:07 PM
To: Chen, Pei
Subject: Re: umls lookup issue

Hi Pei,
we did more debuging and it's the lookup call below (higlighted in yelleow) 
that causes the delay.

performLookup is in DictionaryLookupAnnotator.java

private void performLookup(JCas jcas, LookupSpec ls, List lookupTokenList,
                    Map ctxMap) throws Exception
       {
             // sort the lookup tokens
             Collections.sort(lookupTokenList, 
LookupTokenComparator.getInstance() );

             // perform lookup
             Collection lookupHitCol = null;

             LookupAlgorithm la = (LookupAlgorithm) ls.getLookupAlgorithm();
             lookupHitCol = la.lookup(lookupTokenList, ctxMap);

Samir




From: "Chen, Pei" 
To: "dev@ctakes.apache.org" 
Cc: samir chabou 
Sent: Thursday, August 15, 2013 9:00:37 AM
Subject: RE: umls lookup issue

Hi Samir,
[including the public dev list]
Thanks for opening up a new thread on this issue.
Would you be able to help narrow down the sentence that you believe is causing 
the NP2LookupWindow to take 3h to process?  I can’t seem to reproduce it on my 
end.
I vaguely remember someone running into something where it could go into a 
loop, so hopefully maybe they can also chime in…

--Pei

From: samir chabou [mailto:samir...@yahoo.com]
Sent: Wednesday, August 14, 2013 7:30 PM
To: Chen, Pei
Subject: Re: umls lookup issue


specifically the NP2LookupWindow that causes de delay

From: samir chabou 
To: "Chen, Pei" 
Sent: Wednesday, August 14, 2013 7:21:18 PM
Subject: Re: umls lookup issue


Hi Pei
I removed the LookupWindowAnnotator went very fast less than 1 min but there 
was no annotations for EntityMention and EventMention, it looks there is some 
thinh wrong with the LookupWindowAnnotator
Samir




From: samir chabou 
To: "Chen, Pei" 
Sent: Wednesday, August 14, 2013 7:11:57 PM
Subject: Re: umls lookup issue

Hi Pei
I removed the lookupwindowannotation went very fast less than 1 min but there 
was no annotations for EntityMention and EventMention, it looks there is some 
thinh wrong with the lookupwindowannotation
Samir



From: "Chen, Pei" 
To: samir chabou 
Sent: Wednesday, August 14, 2013 3:40:46 PM
Subject: RE: umls lookup issue

That is strange- it shouldn’t take that long.  I wonder if it’s going into an 
infinite loop.
Have you tried debugging it?  Perhaps removing some of the lines in the note or 
removing the dictionary lookup component itself?
--Pei

From: samir chabou [mailto:samir...@yahoo.com]
Sent: Wednesday, August 14, 2013 1:14 PM
To: Chen, Pei
Subject: Re: umls lookup issue

Hi Pei,
Unfortunately, the removal of the DependencyParsser and Assertion did not make 
difference (it has been running now for 1h so i stopped). Pei I think the 
bottle neck was the LookupWindowAnnotator, yesterday when it was running the 
console showed the LookupWindowAnnotator annotations it took quit time to go 
from one LookupWindow to an other, also these annotations of lookupwindows was 
done twice.

Memory: Xms500M and Xmx1500

The jdk : JavaSE-1.6 (jre7)

below screen capture showing from where i got the memory and jdk info + the 
structure of AggregatePlaintextUMLSProcessor.xml without the DependencyParsser 
and Assertion

Thanks a lot
Samir


From: "Chen, Pei" 
To: samir chabou 
Sent: Wednesday, August 14, 2013 10:08:00 AM
Subject: RE: umls lookup issue

Hi Samir,
It shouldn’t take 3h… it’s a bit strange.  cTAKES is much more constrained to 
memory rather than cpu.  Do you know which JDK and what the java memory 
settings were used?
Could you also try removing the new annotators that were added in 3.0? 
DependencyParser, Assertion Module.  See attached as an example.
--Pei

From: samir chabou [mailto:samir...@yahoo.com]
Sent: Tuesday, August 13, 2013 10:48 PM
To: Chen, Pei
Subject: Re: umls lookup issue

Hi Pei
I tr

Re: umls lookup issue

[samir chabou]

Hi Pei
1- abstract attached file is what I used as a sample 
2- AggregatePlaintextUMLSProcessor attached file is the .xml configuration 
(note: even if i removed the dependencyParser, SemanticRoleLabler, 
AssertionAnnotator and ExtractionPrepAnnotator these does not change the 
performance)

Thank you very much for your help







 From: "Chen, Pei" 
To: samir chabou  
Cc: "dev@ctakes.apache.org"  
Sent: Thursday, August 15, 2013 10:23:50 PM
Subject: RE: umls lookup issue 
 

Hi Samir,
Do you have a sample sentence that causes the 3hr run?
Also could you attach the AggregatePipeline.xml configuration used? In case, 
someone else on the dev list may have encountered this in the past already.

I'll try and see if I can recreate it.
--Pei
________
From: samir chabou [samir...@yahoo.com]
Sent: Thursday, August 15, 2013 7:07 PM
To: Chen, Pei
Subject: Re: umls lookup issue

Hi Pei,
we did more debuging and it's the lookup call below (higlighted in yelleow) 
that causes the delay.

performLookup is in DictionaryLookupAnnotator.java

private void performLookup(JCas jcas, LookupSpec ls, List lookupTokenList,
                    Map ctxMap) throws Exception
       {
             // sort the lookup tokens
             Collections.sort(lookupTokenList, 
LookupTokenComparator.getInstance() );

             // perform lookup
             Collection lookupHitCol = null;

             LookupAlgorithm la = (LookupAlgorithm) ls.getLookupAlgorithm();
             lookupHitCol = la.lookup(lookupTokenList, ctxMap);

Samir




From: "Chen, Pei" 
To: "dev@ctakes.apache.org" 
Cc: samir chabou 
Sent: Thursday, August 15, 2013 9:00:37 AM
Subject: RE: umls lookup issue

Hi Samir,
[including the public dev list]
Thanks for opening up a new thread on this issue.
Would you be able to help narrow down the sentence that you believe is causing 
the NP2LookupWindow to take 3h to process?  I can’t seem to reproduce it on my 
end.
I vaguely remember someone running into something where it could go into a 
loop, so hopefully maybe they can also chime in…

--Pei

From: samir chabou [mailto:samir...@yahoo.com]
Sent: Wednesday, August 14, 2013 7:30 PM
To: Chen, Pei
Subject: Re: umls lookup issue


specifically the NP2LookupWindow that causes de delay
____
From: samir chabou 
To: "Chen, Pei" 
Sent: Wednesday, August 14, 2013 7:21:18 PM
Subject: Re: umls lookup issue


Hi Pei
I removed the LookupWindowAnnotator went very fast less than 1 min but there 
was no annotations for EntityMention and EventMention, it looks there is some 
thinh wrong with the LookupWindowAnnotator
Samir




From: samir chabou 
To: "Chen, Pei" 
Sent: Wednesday, August 14, 2013 7:11:57 PM
Subject: Re: umls lookup issue

Hi Pei
I removed the lookupwindowannotation went very fast less than 1 min but there 
was no annotations for EntityMention and EventMention, it looks there is some 
thinh wrong with the lookupwindowannotation
Samir



From: "Chen, Pei" 
To: samir chabou 
Sent: Wednesday, August 14, 2013 3:40:46 PM
Subject: RE: umls lookup issue

That is strange- it shouldn’t take that long.  I wonder if it’s going into an 
infinite loop.
Have you tried debugging it?  Perhaps removing some of the lines in the note or 
removing the dictionary lookup component itself?
--Pei

From: samir chabou [mailto:samir...@yahoo.com]
Sent: Wednesday, August 14, 2013 1:14 PM
To: Chen, Pei
Subject: Re: umls lookup issue

Hi Pei,
Unfortunately, the removal of the DependencyParsser and Assertion did not make 
difference (it has been running now for 1h so i stopped). Pei I think the 
bottle neck was the LookupWindowAnnotator, yesterday when it was running the 
console showed the LookupWindowAnnotator annotations it took quit time to go 
from one LookupWindow to an other, also these annotations of lookupwindows was 
done twice.

Memory: Xms500M and Xmx1500

The jdk : JavaSE-1.6 (jre7)

below screen capture showing from where i got the memory and jdk info + the 
structure of AggregatePlaintextUMLSProcessor.xml without the DependencyParsser 
and Assertion

Thanks a lot
Samir


From: "Chen, Pei" 
To: samir chabou 
Sent: Wednesday, August 14, 2013 10:08:00 AM
Subject: RE: umls lookup issue

Hi Samir,
It shouldn’t take 3h… it’s a bit strange.  cTAKES is much more constrained to 
memory rather than cpu.  Do you know which JDK and what the java memory 
settings were used?
Could you also try removing the new annotators that were added in 3.0? 
DependencyParser, Assertion Module.  See attached as an example.
--Pei

From: samir chabou [mailto:samir...@yahoo.com]
Sent: Tuesday, August 13, 2013 10:48 PM
To: Chen, Pei
Subject: Re: umls lookup issue

Hi Pei
I tried the clinical pipeline as is with no modificatio

Re: to involve in your development group

[samir chabou]

hi,
I have an issue with ctakes 3.0 when I use the DictionaryLookupAnnotatorUMLS, 
it takes hours before getting the annotation for just one line of input text. I 
didn't have this issue with ctakes 2.5. Do you have an idea why this happening 
in ctakes 3.0 ?
Thank you very much





 From: "Mattmann, Chris A (398J)" 
To: "dev@ctakes.apache.org"  
Sent: Wednesday, August 7, 2013 3:15:42 PM
Subject: Re: to involve in your development group
 

Awesome, great news and yes please proceed!

We're here to help.

Cheers,
Chris

++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattm...@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++






-Original Message-
From: , Pei 
Reply-To: "dev@ctakes.apache.org" 
Date: Wednesday, August 7, 2013 10:22 AM
To: "dev@ctakes.apache.org" 
Subject: RE: to involve in your development group

>Thanks Sandeep.
>Looking forward to it!  Feel free to ping us in case you have any
>questions/issues.
>--Pei
>
>> -Original Message-
>> From: sandeep rg [mailto:sandeep.f...@gmail.com]
>> Sent: Wednesday, August 07, 2013 1:21 PM
>> To: dev@ctakes.apache.org
>> Subject: Re: to involve in your development group
>> 
>> sir,
>> thanks pei chen and chris Mattmann  for accepting my proposal for
>> implementing ocr.i have started my work.i will try maximum to go
>>according
>> to the schedule.i will update my every progress to you.
>> 
>> 
>> On Tue, Jul 23, 2013 at 9:26 PM, sandeep rg 
>> wrote:
>> 
>> > thank you Finan sean, for your suggestion,i am now just going through
>> > the JAI,i think it has more features then javaocr..
>> >
>> >
>> >
>> > On Mon, Jul 22, 2013 at 10:22 PM, Mattmann, Chris A (398J) <
>> > chris.a.mattm...@jpl.nasa.gov> wrote:
>> >
>> >> Hi Sandeep,
>> >>
>> >> I'll try and review this today.
>> >>
>> >> Cheers,
>> >> Chris
>> >>
>> >>
>> ++
>> 
>> >> Chris Mattmann, Ph.D.
>> >> Senior Computer Scientist
>> >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> >> Office: 171-266B, Mailstop: 171-246
>> >> Email: chris.a.mattm...@nasa.gov
>> >> WWW:  http://sunset.usc.edu/~mattmann/
>> >>
>> ++
>> 
>> >> Adjunct Assistant Professor, Computer Science Department University
>> >> of Southern California, Los Angeles, CA 90089 USA
>> >>
>> ++
>> 
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> -Original Message-
>> >> From: sandeep rg 
>> >> Reply-To: "dev@ctakes.apache.org" 
>> >> Date: Monday, July 22, 2013 7:04 AM
>> >> To: "dev@ctakes.apache.org" 
>> >> Subject: Re: to involve in your development group
>> >>
>> >> >sir,
>> >> > i have gone through some of the medical record such as
>> >> >bills,patient details etc. most of them are printed using dot matrix
>> >> >printer,which is very hard to extract such type text from scanned
>> >> >images.i have done testing with some professional software such as
>> >> >abbyy fine reader which also
>> >> given
>> >> >a poor output.
>> >> >
>> >> >but sir i have the confidence to do it.but i need more knowledge
>> >> >about image processing capabilities.so can you suggest any one who
>> >> >is good in image processing programming in your team?
>> >> >
>> >> >
>> >> >On Thu, Jul 18, 2013 at 1:22 AM, sandeep rg 
>> >> >wrote:
>> >> >
>> >> >> i hava done sequence diagram and done some small changes,please go
>> >> >>through  it and tell me if any more thing is to be included
>> >> >>
>> >> >>
>> >> >> On Wed, Jul 17, 2013 at 9:37 PM, sandeep rg
>> >> >>wrote:
>> >> >>
>> >> >>> it just a skeleton of original proposal
>> >> >>>
>> >> >>>
>> >> >>> On Wed, Jul 17, 2013 at 9:31 PM, sandeep rg
>> >> >>>wrote:
>> >> >>>
>> >>  the sample work is shared with you both.any more details to be
>> >> included  please tell me.
>> >>  In which,GUI design,schedule and implementation flow chart
>> >> design is to  added which is under construction and will be
>> >> uploaded within few hours.
>> >> 
>> >> 
>> >>  On Wed, Jul 17, 2013 at 7:56 PM, Chen, Pei <
>> >>  pei.c...@childrens.harvard.edu> wrote:
>> >> 
>> >> > pei.stat...@gmail.com
>> >> >
>> >> > > -Original Message-
>> >> > > From: Mattmann, Chris A (398J)
>> >> >[mailto:chris.a.mattm...@jpl.nasa.gov]
>> >> > > Sent: Wednesday, July 17, 2013 10:22 AM
>> >> > > To: dev@ctakes.apache.org
>> >> > > Subject: Re: to involve in your development group
>> >> > >
>> >> >