RE: Disable yTEX and Regression tests on Jenkins

2017-11-08 Thread Gandhi Rajan Natarajan 
Hi Sean,

I can take it up if someone is willing to guide me on this.

Regards,
Gandhi


-Original Message-
From: Finan, Sean [mailto:sean.fi...@childrens.harvard.edu]
Sent: Wednesday, November 08, 2017 9:45 PM
To: dev@ctakes.apache.org
Subject: Disable yTEX and Regression tests on Jenkins

Hi all,

The Jenkins builds have been failing for about a month now because of internal 
Jenkins changes and 'unit' tests in the ctakes-Regression and ctakes-yTEX 
modules.  This is holding up the build for all of our primary clinical-pipeline 
modules.

If anybody can take a look at the problems and fix them please respond to this 
email.  Otherwise I would like to create a jira issue and disable them until 
somebody does have the time to take care of them.  If you have a good reason 
for these tests not being disabled (e.g. we might forget to fix them) please 
state a case.  I do not intend to act unilaterally on this issue.

Please respond by midnight Friday, November 10.

Thank you,

Sean
This email and any files transmitted with it are confidential and intended 
solely for the use of the individual or entity to whom they are addressed. If 
you are not the named addressee you should not disseminate, distribute or copy 
this e-mail. Please notify the sender or system manager by email immediately if 
you have received this e-mail by mistake and delete this e-mail from your 
system. If you are not the intended recipient you are notified that disclosing, 
copying, distributing or taking any action in reliance on the contents of this 
information is strictly prohibited and against the law.

Disable yTEX and Regression tests on Jenkins

2017-11-08 Thread Finan, Sean 
Hi all,

The Jenkins builds have been failing for about a month now because of internal 
Jenkins changes and 'unit' tests in the ctakes-Regression and ctakes-yTEX 
modules.  This is holding up the build for all of our primary clinical-pipeline 
modules.

If anybody can take a look at the problems and fix them please respond to this 
email.  Otherwise I would like to create a jira issue and disable them until 
somebody does have the time to take care of them.  If you have a good reason 
for these tests not being disabled (e.g. we might forget to fix them) please 
state a case.  I do not intend to act unilaterally on this issue.

Please respond by midnight Friday, November 10.

Thank you,

Sean

Re: source code of user installation of cTakes. [EXTERNAL] [SUSPICIOUS]

2017-11-08 Thread Miller, Timothy 
"if any procedure is happened or not."

I think you need a formal definition of this to understand what you're trying 
to do.

As a guess, you want it to be in the past, related to the patient (not a family 
member), and not negated or hypothetical or uncertain, etc.?
This would require multiple components, but most of them should run in the 
temporal pipeline.

Tim

From: Bhagwat Posane 
Sent: Wednesday, November 8, 2017 2:40 AM
To: dev@ctakes.apache.org
Subject: Re: source code of user installation of cTakes. [EXTERNAL] [SUSPICIOUS]

Hi Sean and Tim,

Could you please provide your valuable inputs for below questions:

I am interested in finding the context of clinical notes like if any
procedure is happened or not.
I think TLINKS  values on web API maybe helpful to get the required result,
not sure though. Please confirm

I am not able to find the code to generate TLINKS, could you please help me
locate the code.

On 7 November 2017 at 16:18, Gandhi Rajan Natarajan <
gandhi.natara...@arisglobal.com> wrote:

> Hi Bhagwat,
>
> Glad that you got it working.
>
> Regarding cTAKES functional questions below, I guess Sean or Tim is the
> right person to give more info.
>
> Regards,
> Gandhi
>
> -Original Message-
> From: Bhagwat Posane [mailto:bhagwat.pos...@gmail.com]
> Sent: Tuesday, November 07, 2017 3:22 PM
> To: dev@ctakes.apache.org
> Subject: Re: source code of user installation of cTakes. [EXTERNAL]
> [SUSPICIOUS]
>
> Thanks Gandhi,
>
> I have gone through both the links and able to get the results.
>
> I am interested in finding the context of clinical notes like if any
> procedure is happened or not.
> I think TLINKS  values on web API maybe helpful to get the required
> result, not sure though. Please confirm
>
> I am not able to find the code to generate TLINKS, could you please help
> me locate the code.
>
> And also could anybody please guide me to find context out of the
> note-sentence.
>
> On 3 November 2017 at 19:44, Gandhi Rajan Natarajan <
> gandhi.natara...@arisglobal.com> wrote:
>
> > Hi Bhagwat,
> >
> > I'm not sure if it will help you. But I extended the code available in
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_healthnlp_examples_tree_master_ctakes-2Dtemporal-2Ddemo=DwIBaQ=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h=ONC114Bki6vY6dmCLn3sPjdNegVyawdkxXvYuBFoonI=uGmJ4i-iqGkA8mjvWJoNlSm7QKjHyRhOtcsjI_Hvs70=
> > to get the XML output and parsed the required mentions for me like
> > Procedure, Disorder etc.
> >
> > You can look at the code here - 
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_=DwIBaQ=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h=ONC114Bki6vY6dmCLn3sPjdNegVyawdkxXvYuBFoonI=N1LP2gEWWo6tF84sfvjw_hqRuW1fEdvqkBw7bM3VYzU=
> > gandhirajan/cTAKES/tree/master/cTakes-Rest-Service/
> > src/main/java/com/ctakes/nlp and see if it helps. Cheers.
> >
> > Regards,
> > Gandhi
> >
> >
> > -Original Message-
> > From: Bhagwat Posane [mailto:bhagwat.pos...@gmail.com]
> > Sent: Friday, November 03, 2017 7:39 PM
> > To: dev@ctakes.apache.org
> > Subject: Re: source code of user installation of cTakes. [EXTERNAL]
> > [SUSPICIOUS]
> >
> > Thank you Sean and Tim for your support.
> >
> > I have followed the links given by Sean and overview by Tim  which is
> > pretty helpful to get the better understanding of source code.
> >
> > I have run ctakes-example project that internally executes
> > ClinicalPipelineFactory.getDefaultPipeline() method. This has given
> > results like POS tag, Polarity, etc.
> > Now, I am more interested in finding Procedure, Medication, Drug, etc.
> > Could you please point me to the code file or help with code snippet
> > to capture above terms.
> >
> >
> >
> > On 30 October 2017 at 19:36, Finan, Sean
> >  > >
> > wrote:
> >
> > > Hi Bhagwat,
> > >
> > > If you are interested in the default clinical pipeline, you can look
> > > at the wiki here:
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_CTAKES_=DwIBaQ=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h=ONC114Bki6vY6dmCLn3sPjdNegVyawdkxXvYuBFoonI=sJCh70kfpVGg_2l3D7h6QYndeat50yHva5sNnmhVZbw=
> > > Default+Clinical+Pipeline
> > > For a visual representation of what Tim described.
> > >
> > > The AEs used for the ctakes 4.0 default clinical pipeline are shown
> > > at the bottom of this wiki page: 
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_=DwIBaQ=qS4goWBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU=Heup-IbsIg9Q1TPOylpP9FE4GTK-OqdTDRRNQXipowRLRjx0ibQrHEo8uYx6674h=ONC114Bki6vY6dmCLn3sPjdNegVyawdkxXvYuBFoonI=oN0sRQQgrlsp8j926ayeysmYTVO2kriknuUjfIjlUq8=
> > > confluence/display/CTAKES/Piper+Files
> > > The

RE: source code of user installation of cTakes. [EXTERNAL] [SUSPICIOUS] [SUSPICIOUS]

2017-11-08 Thread Finan, Sean 
Argh,
The apache listserv occasionally ignores my line breaks:

// Times.  There aren't default models, so set specifically add 
BackwardsTimeAnnotator 
classifierJarPath=/org/apache/ctakes/temporal/ae/timeannotator/model.jar

Should be two lines (comment, then command):

// Times.  There aren't default models, so set specifically 

add BackwardsTimeAnnotator 
classifierJarPath=/org/apache/ctakes/temporal/ae/timeannotator/model.jar


The same goes for the "load X  YY=ZZ" statements that look like they are 
the ends of comment lines.


Sean


-Original Message-
From: Finan, Sean [mailto:sean.fi...@childrens.harvard.edu] 
Sent: Wednesday, November 08, 2017 8:09 AM
To: dev@ctakes.apache.org
Subject: RE: source code of user installation of cTakes. [EXTERNAL] 
[SUSPICIOUS] [SUSPICIOUS]

Hi Bhagwat,

The ctakes-temporal module has the components needed to generate tlinks.
Using trunk, the following can be added to the end of the default clinical 
pipeline piper file (or a copy):

load TemporalSubPipe


The TemporalSubPipe.piper file contains the following setup to create events, 
times, time relative to document writing (doctimerel) and tlinks:

///
// Commands and parameters to create a default temporal processing 
sub-pipeline.  This is not a full pipeline.

// 'Generic' Events.  Use addDescription and let the EventAnnotator set itself 
up with defaults.
addDescription EventAnnotator

// Times.  There aren't default models, so set specifically add 
BackwardsTimeAnnotator 
classifierJarPath=/org/apache/ctakes/temporal/ae/timeannotator/model.jar

// DocTimeRel: the relation bin for Events to the Document Creation Time.
add DocTimeRelAnnotator 
classifierJarPath=/org/apache/ctakes/temporal/ae/doctimerel/model.jar

// Event - Time binary relations.
add EventTimeRelationAnnotator 
classifierJarPath=/org/apache/ctakes/temporal/ae/eventtime/model.jar

// Event - Event binary relations.
add EventEventRelationAnnotator 
classifierJarPath=/org/apache/ctakes/temporal/ae/eventevent/model.jar
///

To see the code used for tlink detection, search ctakes for the 
EventTimeRelationAnnotator.java and EventEventRelationAnnotator.java classes.  
All piper "add" commands are followed by a class name.

I have a piper file that I use for testing a 'full' ctakes pipeline:

///
// Advanced Tokenization: Regex sectionization, BIO Sentence Detector (lumper), 
Paragraphs, Lists load AdvancedTokenizerPipeline

// Always need these ...
add ContextDependentTokenizerAnnotator
add POSTagger

// Chunkers
load ChunkerSubPipe

// Default fast dictionary lookup
set minimumSpan=2
load DictionarySubPipe

// Cleartk Entity Attributes (negation, uncertainty, etc.) load 
AttributeCleartkSubPipe

// Entity Relations (degree/severity, anatomical location) load RelationSubPipe

// Temporal (event, time, dtr, tlink)
load TemporalSubPipe

// Coreferences (e.g. patient = he)
load CorefSubPipe

// Html output
add pretty.html.HtmlTextWriter

// XMI output
writeXmis
///


Sean

-Original Message-
From: Bhagwat Posane [mailto:bhagwat.pos...@gmail.com]
Sent: Wednesday, November 08, 2017 2:41 AM
To: dev@ctakes.apache.org
Subject: Re: source code of user installation of cTakes. [EXTERNAL] [SUSPICIOUS]

Hi Sean and Tim,

Could you please provide your valuable inputs for below questions:

I am interested in finding the context of clinical notes like if any procedure 
is happened or not.
I think TLINKS  values on web API maybe helpful to get the required result, not 
sure though. Please confirm

I am not able to find the code to generate TLINKS, could you please help me 
locate the code.

On 7 November 2017 at 16:18, Gandhi Rajan Natarajan < 
gandhi.natara...@arisglobal.com> wrote:

> Hi Bhagwat,
>
> Glad that you got it working.
>
> Regarding cTAKES functional questions below, I guess Sean or Tim is 
> the right person to give more info.
>
> Regards,
> Gandhi
>
> -Original Message-
> From: Bhagwat Posane [mailto:bhagwat.pos...@gmail.com]
> Sent: Tuesday, November 07, 2017 3:22 PM
> To: dev@ctakes.apache.org
> Subject: Re: source code of user installation of cTakes. [EXTERNAL] 
> [SUSPICIOUS]
>
> Thanks Gandhi,
>
> I have gone through both the links and able to get the results.
>
> I am interested in finding the context of clinical notes like if any 
> procedure is happened or not.
> I think TLINKS  values on web API maybe helpful to get the required 
> result, not sure though. Please confirm
>
> I am

RE: source code of user installation of cTakes. [EXTERNAL] [SUSPICIOUS]

2017-11-08 Thread Finan, Sean 
Hi Bhagwat,

The ctakes-temporal module has the components needed to generate tlinks.
Using trunk, the following can be added to the end of the default clinical 
pipeline piper file (or a copy):

load TemporalSubPipe


The TemporalSubPipe.piper file contains the following setup to create events, 
times, time relative to document writing (doctimerel) and tlinks:

///
// Commands and parameters to create a default temporal processing 
sub-pipeline.  This is not a full pipeline.

// 'Generic' Events.  Use addDescription and let the EventAnnotator set itself 
up with defaults.
addDescription EventAnnotator

// Times.  There aren't default models, so set specifically
add BackwardsTimeAnnotator 
classifierJarPath=/org/apache/ctakes/temporal/ae/timeannotator/model.jar

// DocTimeRel: the relation bin for Events to the Document Creation Time.
add DocTimeRelAnnotator 
classifierJarPath=/org/apache/ctakes/temporal/ae/doctimerel/model.jar

// Event - Time binary relations.
add EventTimeRelationAnnotator 
classifierJarPath=/org/apache/ctakes/temporal/ae/eventtime/model.jar

// Event - Event binary relations.
add EventEventRelationAnnotator 
classifierJarPath=/org/apache/ctakes/temporal/ae/eventevent/model.jar
///

To see the code used for tlink detection, search ctakes for the 
EventTimeRelationAnnotator.java and EventEventRelationAnnotator.java classes.  
All piper "add" commands are followed by a class name.

I have a piper file that I use for testing a 'full' ctakes pipeline:

///
// Advanced Tokenization: Regex sectionization, BIO Sentence Detector (lumper), 
Paragraphs, Lists
load AdvancedTokenizerPipeline

// Always need these ...
add ContextDependentTokenizerAnnotator
add POSTagger

// Chunkers
load ChunkerSubPipe

// Default fast dictionary lookup
set minimumSpan=2
load DictionarySubPipe

// Cleartk Entity Attributes (negation, uncertainty, etc.)
load AttributeCleartkSubPipe

// Entity Relations (degree/severity, anatomical location)
load RelationSubPipe

// Temporal (event, time, dtr, tlink)
load TemporalSubPipe

// Coreferences (e.g. patient = he)
load CorefSubPipe

// Html output
add pretty.html.HtmlTextWriter

// XMI output
writeXmis
///


Sean

-Original Message-
From: Bhagwat Posane [mailto:bhagwat.pos...@gmail.com] 
Sent: Wednesday, November 08, 2017 2:41 AM
To: dev@ctakes.apache.org
Subject: Re: source code of user installation of cTakes. [EXTERNAL] [SUSPICIOUS]

Hi Sean and Tim,

Could you please provide your valuable inputs for below questions:

I am interested in finding the context of clinical notes like if any procedure 
is happened or not.
I think TLINKS  values on web API maybe helpful to get the required result, not 
sure though. Please confirm

I am not able to find the code to generate TLINKS, could you please help me 
locate the code.

On 7 November 2017 at 16:18, Gandhi Rajan Natarajan < 
gandhi.natara...@arisglobal.com> wrote:

> Hi Bhagwat,
>
> Glad that you got it working.
>
> Regarding cTAKES functional questions below, I guess Sean or Tim is 
> the right person to give more info.
>
> Regards,
> Gandhi
>
> -Original Message-
> From: Bhagwat Posane [mailto:bhagwat.pos...@gmail.com]
> Sent: Tuesday, November 07, 2017 3:22 PM
> To: dev@ctakes.apache.org
> Subject: Re: source code of user installation of cTakes. [EXTERNAL] 
> [SUSPICIOUS]
>
> Thanks Gandhi,
>
> I have gone through both the links and able to get the results.
>
> I am interested in finding the context of clinical notes like if any 
> procedure is happened or not.
> I think TLINKS  values on web API maybe helpful to get the required 
> result, not sure though. Please confirm
>
> I am not able to find the code to generate TLINKS, could you please 
> help me locate the code.
>
> And also could anybody please guide me to find context out of the 
> note-sentence.
>
> On 3 November 2017 at 19:44, Gandhi Rajan Natarajan < 
> gandhi.natara...@arisglobal.com> wrote:
>
> > Hi Bhagwat,
> >
> > I'm not sure if it will help you. But I extended the code available 
> > in 
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_heal
> > thnlp_examples_tree_master_ctakes-2Dtemporal-2Ddemo=DwIBaQ=qS4go
> > WBT7poplM69zy_3xhKwEW14JZMSdioCoppxeFU=fs67GvlGZstTpyIisCYNYmQCP6r
> > 0bcpKGd4f7d4gTao=Q-UMs2CriAdL_TaKVFqOnSOfqjR05AQWCnwqn6bOrBk=Wyt
> > apFSqEoy1Za7AzcNuuMLQutzXpWTHlzSVl2Gw7zU=
> > to get the XML output and parsed the required mentions for me like 
> > Procedure, Disorder