Re: Next cTAKES release (3.1)?

We released code on using cTAKES to annotate clinical text and SVMs that use the annotations to classify clinical text from the CMC 2007 and I2B2 2008 challenges: We did the cmd 2007 with cTAKES 2.5: https://code.google.com/p/ytex/wiki/WordSenseDisambiguation_V08#Reproducing_results_on_CMC_2007_ch

move ytex annotators to ctakes.apache.org?

Hello All, I'd like to contribute ytex to ctakes. YTEX's main feature is the ability to store *any* ctakes (or uima) annotation in a relational database (in a relational format), and the ability to export these annotations to ML packages (weka, libsvm, matlab, R). All of this is purely declarati

Re: move ytex annotators to ctakes.apache.org?

Oct 3, 2013 at 10:32 AM, Wu, Stephen T., Ph.D. wrote: > This sounds wonderful, Vijay! > But yes, there will probably be an issue with Hibernate and Weka. What's > the best course of action, anyone? > > stephen > > > On 10/3/13 7:50 AM, "vijay garla" wrote: > &

Re: move ytex annotators to ctakes.apache.org?

s of 20?? >> Member, UCLA Medical Imaging & Informatics Lab Member, CA Delegation >> to the House of Delegates of the American Medical Association >> ksa...@ksarma.com >> gchat: ksa...@gmail.com >> linkedin: www.linkedin.com/in/ksarma >> >> >> On

Re: move ytex annotators to ctakes.apache.org?

udes ytex analysis engines * ctakes-ytex-uima-res resources for ytex analysis engines Alternatively, I can add ctakes-ytex-uima and ctakes-ytex-uima-res to existing projects (don't know where they would fit). Best, Vijay On Thu, Oct 3, 2013 at 7:06 PM, vijay garla wrote: > Hi Pei, &

Re: move ytex annotators to ctakes.apache.org?

the functionality?) > > Also, just curious- how big of a code base was this originally? I'm just > thinking about IP Clearance here (if it's required). > ~250 java source files, ~1mb source (didn't do a line count) the original code is ASF 2.0 license. > > > On

Re: Synonymous terms

I'm curious as to how you decide which is 'the' preferred term; maybe this is clear if you are just using SNOMED-CT. However, if you use multiple UMLS vocabularies, there can be multiple (or no) preferred terms among the MRCONSO entries with the same CUI. -vj On Tue, Oct 22, 2013 at 12:22 PM, M

ytex ctakes patches

For the YTEX port, I've taken a few baby steps ... I've filed some jira tickets with patches: CTAKES-253 and CTAKES-252 , more coming soon. I have a question regarding testing: it seems to me that

Re: using umls dictionary lookup offline?

No need to write any new annotator, just point org.apache.ctakes.dictionary.lookup.ae.DictionaryLookupAnnotator at the UMLS HSQL DB. The UMLS password check is a very weak chastity belt. For ytex, we make umls resources available via a UTS password protected website - to get the resources the use

ytex branch

Hello All, I'm close to done with the port of ytex to ctakes. I would like to create branch to commit the changes to for review by the ctakes elders and other developers. I will be adding the following projects: * ctakes-ytex-res - resources * ctakes-ytex - no uima/ctakes dependencies - primaril

Re: ytex branch

- were the type system changes mainly adding additional > fields? Just planning ahead especially for proposed type system changes... > > --Pei > > > -Original Message- > > From: vijay garla [mailto:vnga...@gmail.com ] > > Sent: Monday, November 25, 2013 5:

Re: FW: svn commit: r1551805 - /ctakes/branches/ytex/ctakes-assertion/src/main/java/org/apache/ctakes/assertion/medfacts/i2b2/api/CharacterOffsetToLineTokenConverterCtakesImpl.java

Hi Pei, I'm not sure if that would solve the problem: change in the ytex branch causes newlines to be ignored (i.e. not treated as a token). trunk's sentence splitter is splits sentences on newlines, so newlines would never be found in a sentence. However, if we had a reproducer we could check i

YTEX cTAKES 3.1.1 ready

Hello All, I have finished an initial cut at the port of YTEX to cTAKES 3.1.1. Most of the YTEX functionality has been ported and integrated with cTAKES, and I've tested with MySQL and MS SQL Server (oracle tests pending). Most of the changes were made in new projects - very little existing cTAK

Re: YTEX cTAKES 3.1.1 ready

> Downloading the code now... To be continued... > > [1] > https://repository.apache.org/content/groups/snapshots/org/apache/ctakes/ > [2] http://sourceforge.net/p/ctakesresources/code/HEAD/tree/trunk/ > [3] https://cwiki.apache.org/confluence/display/CTAKES/cTAKES > > >

Re: YTEX cTAKES 3.1.1 ready

.awt.EventQueue$4.run(Unknown Source) > at java.awt.EventQueue$4.run(Unknown Source) > at java.security.AccessController.doPrivileged(Native Method) > at > java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Sour > ce) > at java.awt.EventQue

Re: svn commit: r1551805 - /ctakes/branches/ytex/ctakes-assertion/src/main/java/org/apache/ctakes/assertion/medfacts/i2b2/api/CharacterOffsetToLineTokenConverterCtakesImpl.java

The issue is indeed the sentence splitter - negation is limited to words within the sentence, and if newlines are considered sentence boundaries, it doesn't work properly (splitting on newlines breaks many other things as well). The YTEX branch includes a sentence splitter that does not automatica

Re: sentence splitter & forks/branches

@cchmc.org > 513-803-9032 > > > -Original Message- > From: vijay garla [mailto:vnga...@gmail.com] > Sent: Wednesday, January 15, 2014 11:34 AM > To: dev@ctakes.apache.org > Subject: Re: svn commit: r1551805 - > /ctakes/branches/ytex/ctak

Re: sentence splitter & forks/branches

eral question about forks, specifically the YTEX branch > that > > > Vijay mentions. > > > If I wanted to implement just the sentence splitter from YTEX into a > > > currently existing 3.1 install, how would I do that? Is it possible? > Or do > > > I have

Re: sentence detector newline behavior

The sentence detection opennlp model used by ctakes does not split sentences at newlines - there is additional logic in the takes sentence splitter that does this (and an alternative impl that doesn't is in the ytex branch). Afaik no retraining / change to the feature representation is necessary.

Re: sentence detector newline behavior

r model > which can use a new line as a end-of-sentence character. > > In case you have certain rules to split sentences we should have a look at > them. The Sentence Detector could be extended to support a user provided > rule based splitter. If there is an interest in that we could

Re: sentence detector newline behavior

For clarity, I'd like to stress that the opennlp sentence model distributed with ctakes today does 'work' with sentences that span newlines - as I understand it, this model ignores newline tokens (or newlines are not provided as features to that model). I believe the improvements Tim and others ar

Re: sentence detector newline behavior

n the opennlp model. Vj On Monday, January 27, 2014, Tim Miller < timothy.mil...@childrens.harvard.edu> wrote: > > On 01/27/2014 06:03 PM, vijay garla wrote: > >> For clarity, I'd like to stress that the opennlp sentence model >> distributed >> with ctakes

Re: YTEX cTAKES 3.1.1 ready

install ytex-0.8, or YTEX has been entirely > merged into cTakes? > > If the last statement is correct, there are missing parts in i.e the UMLS > install steps that are linked from the new ctakes 3.1.1 document. > > Thanks, > vlad > > > On Friday, January 3, 2014 10

Re: YTEX cTAKES 3.1.1 ready

set and then just use the resulting > db. Thoughts? > > Thanks, > Vlad Valtchinov > Brigham Rad > > > On Thursday, January 30, 2014 5:17:43 PM UTC-5, vijay garla wrote: > >> Hi Vlad, >> >> > All of ytex has been moved into ctakes, it is currently in a

Re: YTEX cTAKES 3.1.1 ready

013AA (I assume this will complete without issues as long as the umls schema hasn't changed from 2012). what trial and error did you have to go through to build the distro? -vj On Wed, Feb 5, 2014 at 5:33 PM, vijay garla wrote: > Hi Vlad, > > sorry that the instructions aren't

Re: YTEX cTAKES 3.1.1 ready

trunk at this point? > As you mentioned earlier, it's largely complementary. > [I was just thinking of saving effort to maintain the separate branch and > for simplicity for dev...] > > --Pei > > > -Original Message- > > From: vijay garla [mailto:vnga...@gmail.co

Re: YTEX cTAKES 3.1.1 ready

27;t suggesting that it > only support such a structure. > > Where is that easy button? > > -Original Message- > From: vijay garla [mailto:vnga...@gmail.com] > Sent: Thursday, February 06, 2014 10:31 AM > To: dev@ctakes.apache.org > Cc: ytex-us...@googlegroups.com; ctakes-...@i

Re: YTEX LVG Fix

Hi John, Thanks for this. I've updated the YTEXPipeline, fixed the lvg paths in SetupAUIFirstWord. If you want to re-run SetupAUIFirstWord (not necessary unless you are using the stemmed words for dictionary lookup), just svn update, rebuild ctakes-ytex-uima, and copy the jar to the lib dir of y

YTEX Doc in cwiki

Hello All, I've made a first cut at moving and updating the YTEX docs over from google code to the cTAKES confluence site. This is a first cut, and I'm trying to keep the YTEX docs separated, as it is not yet in trunk/released, and I don't want to mess up any existing docs. see https://cwiki.apa

Re: YTEX Doc in cwiki

of the examples on the Semantic+Similarity page use "cd CTAKES_HOME" > but later use %CTAKES_HOME% > > so it looks like you meant cd %CTAKES_HOME% > > I didn't see anything about the Similarity Benchmark on the new pages. Is > that still part of ytex? > > --

Re: YTEX install - one error after building

n links to > our > > > web pages (the children pages) at the bottom. Is your intent to remove > the > > > first 3 links once things are finalized? > > > > > > some of the examples on the Semantic+Similarity page use "cd > CTAKES_HOME" > > >

ytex merged into trunk

Hello All, I have merged YTEX into trunk, will keep the branch around a little while then delete it. Some non-ytex related changes (I will gladly change/revert if there are objections): * ctakes-temporal does not compile; from the email threads I take it that this is under development and the com

Re: ytex merged into trunk

s] > [INFO] Apache cTAKES YTEX .... SUCCESS [ > 14.592 s] > [INFO] Apache cTAKES YTEX UIMA ... FAILURE [01:34 > min] > [INFO] Apache cTAKES ctakes-clinical-pipeline SKIPPED > [INFO] Apache cTAKES YTEX Web ...

Re: ytex merged into trunk

l -pl ctakes-ytex (executed from ctakes root dir) This is the plugin that maven is complaining about: org.apache.maven.plugins maven-dependency-plugin copy-dependencies compile copy-dependencies ${basedir}/target/lib false false true On Mon, Apr 28, 2014 at 1:26 PM, vijay garla wrote: &g

Re: Ctakes-data-vis

I think one major issue with ctakes in a web server is thread safety. I know that LVG is not thread safe, and it isn't clear what the status is on other components. On Tue, Apr 29, 2014 at 9:20 AM, John Green wrote: > Pei - I meant as a web app, can we keep the credentials loaded and the > reso

Re: resources in ctakes jars

I am all for adding resources to the resource jars. It makes it very difficult to write code/tests that use other modules when the resources can't be resolved using normal maven dependencies. On Fri, May 2, 2014 at 12:30 PM, Chen, Pei wrote: > There is a filter in the root pom.xml to only inclu

Re: Missing artifact org.apache.ctakes:ctakes-ytex-res:jar:3.1.2-SNAPSHOT

You should to do a full build (from the root pom); the ctakes-ytex-res will get built and installed in maven. Alternatively, just build the ctakes-ytex-res project On Thu, May 8, 2014 at 5:05 PM, wrote: > Hi, > > I synchronised my copy of project with the repository and since then I > cannot e

Re: Preparing for an Apache cTAKES 3.2 Release?

regardless of the name, I think it would be incredibly helpful to have thorough documentation on the dictionary lookup, how to configure it, and how to create new dictionaries. I would venture to say that this is the most important component in cTAKES, and probably the one that has generated the m

Re: Building

Hi John, YTEX ships with 3 concept graphs (see https://cwiki.apache.org/confluence/display/CTAKES/cTAKES+3.1.2+-+Semantic+Similarity ): - sct-rxnorm: concepts from SNOMED-CT and RXNORM. This is the default. - sct-msh-csp-aod: concepts from the SNOMED-CT, MeSH, CRISP, and Alcohol and Dru

Re: cTAKES 3.2 Analysis Batch Issue

Hi clayton, I assume you are running the fracture_demo.xml cpe - is that correct? The CPE GUI should give you the option to set the analysis batch. (see attached screenshot). That being said, the analysis_batch is not required (it will default to the current date). Can you attach the log file?

Re: Building

r the ctakes components part (sorry, thanks for pointing me there, > ill > > get to work on reading it). > > > > So bottom line: are the ones that shipped watered down versions? And if > > not, why are my concepts coming up short? If you give me a hint at where >

Re: Building

ork on reading it). > > So bottom line: are the ones that shipped watered down versions? And if > not, why are my concepts coming up short? If you give me a hint at where to > check Ill investigate. > > Thanks! > JG > > > On Sun, Jun 29, 2014 at 8:56 PM, vijay gar

Re: Building

ploy it this > way local. > > Thanks again, > JG > > > On Wed, Jul 2, 2014 at 10:58 PM, vijay garla > wrote: > > > The ytexWeb application tries to look up concepts from terms using the > ytex > > dictionary lookup table, which is a small subset of the UML

Re: cTAKES 3.2 Analysis Batch Issue

L database (umls installed - not sure if that can >> be related). >> >> Here's a screenshot of my CPE (using ytex): >> [image: Inline image 1] >> >> >> >> >> On Wed, Jul 2, 2014 at 10:48 PM, vijay garla wrote: >> >>> Hi clayton,

Re: Converting UMLS codes to SNOMED codes using the YTEX pipeline

caveat: this may have changed with the new dictionary lookup algorithm, but fwiw this is base don the old dictionary lookup The simple answer to 'why no snomed codes by default with ytex?' this is a cleaner data model, and a faster dictionary lookup. If you want to map cuis to snomed codes, simpl

Re: cTAKES 3.2 Analysis Batch Issue

CPE. When running the CVD I have > access to a log file within the gui, but that does not seem to be present > here. Is there a specific place that this log file is saved? > > > On Tue, Jul 8, 2014 at 3:14 AM, vijay garla wrote: > > > Hi Clayton, > > > > The scr

Re: [VOTE] Release Apache cTAKES 3.2.0 (rc2)

ctakes-ytex-lib-3.1.2-SNAPSHOT.zip - this contains non-asf compliant ytex libs. I would like to add it to the sourceforge site / or add it to the ctakes resources directly (that way users simply have to unzip a single zip file)

Re: [VOTE] Release Apache cTAKES 3.2.0 (rc2)

terms, not > SNOMED-CT or the other sources taken from UMLS. If that is not correct, > then I agree there is a problem with what is being shipped in lucene > indexes. > > -- James > > -Original Message- > From: vijay garla [mailto:vnga...@gmail.com] > Sent: Wednesday

Re: DBConsumer

You can add the DBConsumer to any pipeline, or add it to any CPE config. See https://cwiki.apache.org/confluence/display/CTAKES/cTAKES+3.1.2+-+YTEX+DBConsumer You will have to set up ctakes to and your database as documented here: https://cwiki.apache.org/confluence/display/CTAKES/YTEX+Installati

Re: DBConsumer

> Thank you for your time Vijay,JG > > > — > Sent from Mailbox for iPhone > > On Tue, Jul 15, 2014 at 8:26 AM, vijay garla wrote: > > > You can add the DBConsumer to any pipeline, or add it to any CPE config. > > See > > > https://cwiki.apache.org

Re: cTAKES CPE MySQL Exception

My guess is that this exception is coming out of the DictionaryLookup (it creates a connection and holds on to it for the life of the AE). If it is coming out of the DBCollectionReader/DBConsumer you're in luck, as those use a connection pool, and you can configure it to check the connection upon

Re: ytex examples

Hi Clayton, Haven't gotten around to upgrading the docs look here for examples: https://code.google.com/p/ytex/source/browse/#svn%2Ftrunk%2Fworkspace%2Fexamples%2Ffracture If you are using R/Matlab/Python it is easy to generate a sparse matrix directly via database queries, I can give you a few e

Re: Exporting YTEX Pipeline

Can you try this: copy https://code.google.com/p/ytex/source/browse/trunk/workspace/examples/fracture/cui/export.template.xml to CTAKES_HOME\desc\ctakes-ytex\fracture\cui.xml replace %DB_SCHEMA% with your database schema name (value of db.schema in your ytex.properties file) Then from a command pr

Re: Exporting YTEX Pipeline

/... > > Thank you so much for putting up with my issues > > -Clayton > > > On Wed, Jul 30, 2014 at 2:48 PM, vijay garla wrote: > > > Can you try this: > > copy > > > > > https://code.google.com/p/ytex/source/browse/trunk/workspace/examples/fracture

Re: Change from SNOMEDCT to SNOMEDCT_US affecting v_snomed_fword_lookup

That would definitely make sense On Thursday, August 21, 2014, Chen, Pei wrote: > VJ, > Would it make sense to add in('SNOMEDCT_US') to the default > ctakes-ytex/scripts/data/**/insert_view.sql? > That way it'll support umls2011 as well as the newer 2014 naming > conventions? > Ex: inner join MR

Re: Acronym annotator

This is a type of word sense disambiguation; there is a lot of literature on this subject. Co-occurence is one way of doing it, not necessarily the best; you need a ton of annotated data for it to work well. On Thu, Aug 21, 2014 at 9:08 PM, John Green wrote: > Are there any acronym annotators

Re: org.apache.ctakes.ytex.umls.dao.UMLSDaoTest

That is an expected error having to do with the fact that UMLS isn't installed in the test database that get's fired up for unit tests. That is actually a warning (and should be interpreted as an error only if you do have UMLS set up) On Mon, Aug 25, 2014 at 9:02 PM, Pei Chen wrote: > Hi VJ, >

Re: managing ctakes resources on classpath

Hi The test that is failing has nothing to do with the MRCONSO not found warning. ValidationTest failed because it couldn't find the XSD. The XSD is in the ctakes-ytex-resources, but the corresponding maven artifact is an empty jar. I think it would be best to modify the resource jars to actual

Re: sentence detector model

Why not use the i2b2 corpora? On Monday, September 29, 2014, Dligach, Dmitriy < dmitriy.dlig...@childrens.harvard.edu> wrote: > Maybe creating a made-up set of sentences would be an option? That way we > could agree on the annotation of concrete cases. Although this would be > more of a unit test

Re: NPE with ytex in ctakes 3.2.0

The error is caused by not finding the required properties files/xml config files. There are some issues with the ytex setup scripts for the 3.2 release; I have fixed that in trunk. I am updating the 3.2 installation guide with the patched setup scripts. It's not clear to me if you're running fro

Re: YTEX Semantic Sim RESTful

Hi John, Looking at the code, that error is due to the concept graph 'umls' not being loaded. by default, ytex is configured to use the sct-rxnorm concept graph. Can you see if this works: http://localhost:8080/services/rest/similarity?conceptGraph=sct-rxnorm&concept1=C0018787&concept2=C0024109&

Re: YTEX depends on trove4j? LGPL issue

YTEX has several non ASF compliant dependencies, and I thought I had reviewed them and excluded them from the cTAKES distro. The non-ASF YTEX dependencies should be excluded in the ctakes-distribution\src\main\assembly\bin.xml We distribute the non ASF dependencies similarly to ctakes resources (i.

Re: YTEX semantic similarity concept graph questions

I don't know what the difference between PAR/CHD (parent/child) and RB/RN (broader/narrower) is supposed to be. some umls source vocabularies use PAR/CHD only/predominantly (e.g. SNOMED-CT), others use RB/RN (e.g. RXNORM). You can use and experiment with whatever relationships you want (I think t

Re: Errors trying to build ctakes from trunk

This is code generated by the jaxb maven plugin. When you run the maven build, the source is generated under ctakes-ytex\target\generated-sources\jaxb. The maven plugin should automatically add this path to the compile source paths -vj On Sun, Nov 23, 2014 at 3:50 AM, David Kincaid wrote: > I

Re: [VOTE] Release Apache cTAKES 3.2.1 (rc2)

+1 Release the packages as Apache cTAKES 3.2.1

Re: cTakes polarity problem

As guergana mentioned ctakes has a rule based negation detection module. In addition ytex adds a negex based analysis engine. Both approaches are very sensitive to sentence splitting (see previous threads on alternative sentence splitters). An additional advantage of rule based negation is you do

Re: Negex

I think the original ctakes negation AE is in the spirit of Negex, but it is not Negex. AFAICT the ctakes negation AE * requires that triggers are single tokens * does not support conjuctions (e.g. however, nevertheless) or post-negation triggers (e.g. free, was ruled out) * is based on a FSM, whi

Re: Question about YTEX

/LookupDesc_SNOMED.template.xml I don't know why there was an issue extracting ctakes-ytex-res-3.2.2.jar. Can you make sure that /usr/local/apache-ctakes-3.2.2/resources exists and is writable? -vj On Wed, Jun 10, 2015 at 12:17 AM, Tsung-Ting Kuo wrote: > Hi Vijay Garla, > > > > I am Tsun

Re: Question about YTEX

ctionary lookup table with all > concepts from the UMLS successfully? > > > > Thanks very much! > > > > Best regards, > > Tim > > > > *From:* Tsung-Ting Kuo [mailto:ts...@ucsd.edu] > *Sent:* Wednesday, June 10, 2015 9:49 AM > *To:* 'Vijay Garla&

Re: Question about YTEX

I am seeking for YTEX’s help)? I saw > “ctakes-ytex-web-3.2.2-classes.jar” in the “lib” folder, but I am not sure > how to deploy it as a web service. > > > > Best regards, > > Tim > > > > > > *From:* Vijay Garla [mailto:vijay.ga...@yale.edu] > *Sent:* W

Re: Question about YTEX

gt; database, how could I use it in the “Semantic Search” function? > > > > Thanks very much! > > > > Best regards, > > Tim > > > > *From:* Tsung-Ting Kuo [mailto:ts...@ucsd.edu] > *Sent:* Thursday, June 11, 2015 11:10 AM > *To:* 'Vijay Garla' &g

Re: The SegmentRegexAnnotator of Ytex

see https://cwiki.apache.org/confluence/display/CTAKES/User%27s+Guide best, vj On Mon, Jul 13, 2015 at 2:50 AM, Oranit Dror wrote: > Hello, > > I am using ctakes 3.2.2. and recently I have tried to apply the YTEX > pipeline. Particularly, I am interested in the SegmentRegexAnnotator of > Ytex.

Re: The SegmentRegexAnnotator of Ytex

ror wrote: > Thank you, Vijay. > However, I am still encountering with the crash. > > Best, > Oranit. > > -Original Message- > From: vijay garla [mailto:vnga...@gmail.com] > Sent: Monday, July 13, 2015 5:53 PM > To: dev@ctakes.apache.org > Subject: Re: The