>is the purpose of a CasConsumer to essentially save your data Correct, though it is a generic (and archaic) term indicating any end-user of the cas.
> -----Original Message----- > From: clayclay...@gmail.com [mailto:clayclay...@gmail.com] On Behalf Of > Clayton Turner > Sent: Wednesday, August 13, 2014 2:10 PM > To: dev@ctakes.apache.org > Subject: Re: v_snomed_fword_lookup view > > Oh okay, so is the purpose of a CasConsumer to essentially save your data in a > representation that you can do some kind of data mining or classification on > it? > If so, then I think I need to look into making/using one of those. > > > On Wed, Aug 13, 2014 at 1:41 PM, Finan, Sean < > sean.fi...@childrens.harvard.edu> wrote: > > > Hi Clayton, > > > > I'm glad that you got it working. Though I stated that I would, I > > haven't yet checked the fidelity of trunk. Urgent data request one > > day, "must have" writing the next ... and I still live with the > > delusion that I left academia to have free time ... > > > > I have never used ytex or weka, so I'm unfamiliar with all things .arff . > > Could it be that the ytex .arff exporter needs to change consumed > > cTakes annotation classes (>3.1)? > > > > I have a custom CasConsumer that saves text spans and Cuis to file in > > a simple list, and that is what I used for the performance analysis of > > the lookup module. For our other projects here in Beantown we have > > other various outputs that fit the job at hand: text flat files, xml > > files, sql database tables, knot-encoded lace doilies, etc. > > > > I'm sure that none of the above helps you, but I felt obliged to > > provide some kind of answer to your question. > > > > Sean > > > > > -----Original Message----- > > > From: clayclay...@gmail.com [mailto:clayclay...@gmail.com] On Behalf > > > Of Clayton Turner > > > Sent: Wednesday, August 13, 2014 12:25 PM > > > To: dev@ctakes.apache.org > > > Subject: Re: v_snomed_fword_lookup view > > > > > > Okay, I believe I have ctakes dictionary fast working now. Something > > > I'm > > curious > > > about, though, is how you extract the data in order to conduct analysis. > > > > > > I've, in the past, been using the SparseDataExporterImpl from ytex > > > in > > order to > > > create a .arff file for use in weka, but the ctakes pipeline I'm > > > using > > doesn't seem > > > to be compatible with this ytex exporting as I'm not getting any > > > cuis in > > my arff > > > file. > > > > > > I'm using the aggregate plain text umls processor analysis engine > > > from > > ctakes > > > and then using the dbconsumer analysis engine from ytex (for storing > > into the > > > database with regard to analysis batch). > > > > > > Any tips for exporting or some simple issue I'm missing? > > > > > > Thanks, > > > Clayton > > > > > > > > > On Mon, Aug 11, 2014 at 2:09 PM, Harpreet Khanduja <hsk5...@rit.edu> > > > wrote: > > > > > > > Yes, absolutely and > > > > no problem at all. > > > > > > > > Regards, > > > > Harpreet > > > > > > > > > > > > On Mon, Aug 11, 2014 at 1:16 PM, Finan, Sean < > > > > sean.fi...@childrens.harvard.edu> wrote: > > > > > > > > > Thanks Harpreet, > > > > > That is definitely necessary to build! > > > > > > > > > > Those lines should already be in the pom, but commented out. I > > > > > think > > > > that > > > > > some version/branching issues may have arisen at some point wrt > > > > > this > > > > module > > > > > ... > > > > > > > > > > If somebody beats me to it then cheers, otherwise I will try to > > > > > check out tonight and get all the bits in place. > > > > > > > > > > Sean > > > > > > > > > > > -----Original Message----- > > > > > > From: Harpreet Khanduja [mailto:hsk5...@rit.edu] > > > > > > Sent: Monday, August 11, 2014 1:12 PM > > > > > > To: dev@ctakes.apache.org > > > > > > Subject: Re: v_snomed_fword_lookup view > > > > > > > > > > > > Hello Clayton, > > > > > > I do not know about ytex, but I did switch from > > > > > > dictionary-lookup to > > > > > dictionary- > > > > > > lookup-fast. > > > > > > I update my ctakes-dictionary-lookup-fast project using maven. > > > > > > I think I used Team- Update and switched to the latest > > > > > > revision > > > > > available and > > > > > > then > > > > > > I downloaded new 3.2 resources from the for umls. and then I > > > > > > added > > > > > these > > > > > > resources to my > > > > > > ctakes-dictionary-lookup-fast resources folder and also the > > > > > > classpath > > > > > in ctakes- > > > > > > clinical-pipeline. > > > > > > > > > > > > Then I changed the pom.xml file which belongs to the whole > > > > > > ctakes > > > > > project and > > > > > > added <dependency> <groupId>org.apache.ctakes</groupId> > > > > > > <artifactId>ctakes-dictionary-lookup-res</artifactId> > > > > > > <version>${ctakes.version}</version> > > > > > > </dependency> > > > > > > <dependency> > > > > > > <groupId>org.apache.ctakes</groupId> > > > > > > <artifactId>ctakes-dictionary-lookup-fast</artifactId> > > > > > > <version>${ctakes.version}</version> > > > > > > </dependency> > > > > > > > > > > > > > > > > > > these two dependencies to the file. > > > > > > > > > > > > > > > > > > After this, I also added the dependency > > > > > > <dependency> > > > > > > <groupId>org.apache.ctakes</groupId> > > > > > > <artifactId>ctakes-dictionary-lookup-fast</artifactId> > > > > > > </dependency> > > > > > > > > > > > > to the pom.xml of ctakes-clinical-pipeline. > > > > > > > > > > > > And then add the resources folder in ctakes-clinical-pipeline > > > > > > using > > > > > build path > > > > > > configuration under "add class" option. > > > > > > > > > > > > After this it should work. > > > > > > > > > > > > > > > > > > Regards, > > > > > > Harpreet > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Aug 11, 2014 at 12:44 PM, Clayton Turner > > > > > > <caturn...@g.cofc.edu > > > > > > > > > > > wrote: > > > > > > > > > > > > > I still get the same error with the ctakes3.2 branch. Any > > > > suggestions? > > > > > > > > > > > > > > > > > > > > > On Mon, Aug 11, 2014 at 12:06 PM, Clayton Turner > > > > > > > <caturn...@g.cofc.edu> > > > > > > > wrote: > > > > > > > > > > > > > > > I'm going to do a clean install through the repo rather > > > > > > > > than the binaries and see if that fixes my issue because I > > > > > > > > think I just read a past post saying the lookup2 folders exist > > > > > > > > there. > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Aug 11, 2014 at 11:52 AM, Clayton Turner > > > > > > > > <caturn...@g.cofc.edu> > > > > > > > > wrote: > > > > > > > > > > > > > > > >> When navigating to > > > > > > > >> ctakes-dictionary-lookup-fast\desc\analysis_engine > > > > > > > >> there are 2 files, assumedly analysis engines. > > > > > > > >> > > > > > > > >> SnomedLookupAnnotator.xml and > SnomedOvLookupAnnotator.xml > > > > > > > >> > > > > > > > >> If I pick either, I put in my UMLS information but > > > > > > > >> receive an > > > > error > > > > > > > >> when trying to run the CPE: > > > > > > > >> > > > > > > > >> Initialization of CAS Processor with name > > > > "SnomedOvLookupAnnotator" > > > > > > > >> failed. > > > > > > > >> CausedBy: > > > org.apache.uima.resource.ResourceConfigurationException: > > > > > > > >> Initialization of CAS processor with name > > > > "SnomedOvLookupAnnotator" > > > > > > > >> failed. > > > > > > > >> CausedBy: > > > > org.apache.uima.resource.ResourceInitializationException: > > > > > > > Error > > > > > > > >> initializing "org.apache.uima.resource.impl.DataResource_impl" > > > > from > > > > > > > >> descriptor file:..............SnomedLookupAnnotator.xml > > > > > > > >> CausedBy: > > > > org.apache.uima.resource.ResourceInitializationException: > > > > > > > Could > > > > > > > >> not > > > > > > > >> access the resource data at > > > > > > > >> > > > > > > > >> > > > > > > > > > > > file:org\apache\ctakes\dictionary\lookup2\Snomed2011ab_ctakesTui\c > > > > Take > > > > > > > sSnomed.xml > > > > > > > >> > > > > > > > >> Now, I don't even have a "lookup2" folder and, > > > > > > > >> subsequently the > > > > Tui > > > > > > > >> folder and cTakesSnomed.xml file. This seems to be the > > > > > > > >> problem, > > > > but > > > > > > > >> I'm > > > > > > > not > > > > > > > >> sure where these files are supposed to be grabbed from. > > > > > > > >> > > > > > > > >> > > > > > > > >> On Mon, Aug 11, 2014 at 11:47 AM, Clayton Turner > > > > > > > >> <caturn...@g.cofc.edu> > > > > > > > >> wrote: > > > > > > > >> > > > > > > > >>> Hi again: > > > > > > > >>> > > > > > > > >>> How exactly do you switch to using the cTakes > > > > > dictionary-lookup-fast. > > > > > > > Do > > > > > > > >>> I need to go in and alter xml files or is it as simple > > > > > > > >>> as adding > > > > a > > > > > > > certain > > > > > > > >>> item to the list of analysis engines? > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> On Fri, Aug 8, 2014 at 3:48 PM, Finan, Sean < > > > > > > > >>> sean.fi...@childrens.harvard.edu> wrote: > > > > > > > >>> > > > > > > > >>>> Hi Clayton, > > > > > > > >>>> > > > > > > > >>>> I don't know how the ytex dictionary lookup works, so > > > > > > > >>>> I'm afraid that > > > > > > > I > > > > > > > >>>> can't help you with an answer. Maybe Vijay is the best > > > > > > > >>>> person > > > > to > > > > > > > >>>> do > > > > > > > this. > > > > > > > >>>> If you aren't tied to ytex you could try the new > > > > > > > >>>> cTakes dictionary-lookup-fast. I tested "Patient came > > > > > > > >>>> in with a malar > > > > > rash" > > > > > > > and > > > > > > > >>>> it found "malar" and "malar rash". > > > > > > > >>>> > > > > > > > >>>> Vijay, > > > > > > > >>>> > > > > > > > >>>> At some point the lookup-fast module will be the > > > > > > > >>>> default for the > > > > > > > cTakes > > > > > > > >>>> clinical pipeline. In order to synchronize the ytex > > > > > > > >>>> lookup with > > > > > > > cTakes, > > > > > > > >>>> would you like to eventually work together on reusing > > > > > > > >>>> the same code > > > > > > > for > > > > > > > >>>> ytex? I have no idea what ytex does, but I know the > > > > > > > >>>> ins and > > > > outs > > > > > > > >>>> of > > > > > > > the > > > > > > > >>>> cdl-fast module. > > > > > > > >>>> > > > > > > > >>>> Sean > > > > > > > >>>> > > > > > > > >>>> > -----Original Message----- > > > > > > > >>>> > From: clayclay...@gmail.com > > > > > > > >>>> > [mailto:clayclay...@gmail.com] On > > > > > > > Behalf > > > > > > > >>>> Of > > > > > > > >>>> > Clayton Turner > > > > > > > >>>> > Sent: Friday, August 08, 2014 2:08 PM > > > > > > > >>>> > To: dev@ctakes.apache.org > > > > > > > >>>> > Subject: v_snomed_fword_lookup view > > > > > > > >>>> > > > > > > > > >>>> > Hi Everyone: > > > > > > > >>>> > > > > > > > > >>>> > I have a question about how the v_snomed_fword_lookup > > > > > > > >>>> > view works > > > > > > > when > > > > > > > >>>> > running the CPE. > > > > > > > >>>> > > > > > > > > >>>> > So my understanding of the view is that it is a view > > > > > > > >>>> > comprised of > > > > > > > the > > > > > > > >>>> > ytex.umls_aui_fword table, the umls.mrconso table and > > > > > > > >>>> > bits/pieces > > > > > > > from > > > > > > > >>>> > other umls tables. > > > > > > > >>>> > > > > > > > > >>>> > I feel like this is not completely correct or my idea > > > > > > > >>>> > of how the > > > > > > > join > > > > > > > >>>> to > > > > > > > >>>> > create the view works is off. For example, let's say > > > > > > > >>>> > I want > > > > the > > > > > > > >>>> > CPE > > > > > > > >>>> to find > > > > > > > >>>> > "malar ____" (e.g. malar rash) as a concept in the > > > > annotations. > > > > > > > >>>> > It > > > > > > > >>>> never > > > > > > > >>>> > happens after running my CPE descriptor and I cannot > > > > > > > >>>> > find it > > > > in > > > > > > > >>>> > my v_snomed_fword_lookup view. > > > > > > > >>>> > > > > > > > > >>>> > select count(*) from umls_aui_fword where > > > > > > > >>>> > fword='malar'; > > > > yields > > > > > > > >>>> > 34 > > > > > > > >>>> results > > > > > > > >>>> > > > > > > > > >>>> > select count(*) from umls.mrconso where str='malar'; > > > > > > > >>>> > yields 3 > > > > > > > results. > > > > > > > >>>> > > > > > > > > >>>> > So clearly these two tables know what the cui and > > > > > > > >>>> > context(s) are for > > > > > > > >>>> malar > > > > > > > >>>> > ____. Yet, whenever I run a gold standard set of > > > > > > > >>>> > notes through the > > > > > > > >>>> CPE, > > > > > > > >>>> > malar is constantly flagged as just a word token and > > > > > > > >>>> > the concept is > > > > > > > >>>> never > > > > > > > >>>> > grabbed. This is recurrent for lots of other > > > > > > > >>>> > concepts, as > > > > well, > > > > > > > >>>> > I > > > > > > > just > > > > > > > >>>> > wanted to use an example to illustrate my issue. > > > > > > > >>>> > > > > > > > > >>>> > Some troubleshooting I already went through: > > > > > > > >>>> > 1) Reinstalled ytex and umls database objects > > > > > > > >>>> > 2) Reinstalled a second time after redownloading umls > > > > > > > >>>> > through metamorphosys, ensuring that snomed > > > > > > > >>>> > vocabularies were included (also checked file sizes > > > > > > > >>>> > and noticed a big difference so I > > > > know > > > > > > > >>>> > those vocabularies ARE included > > > > > > > >>>> > > > > > > > > >>>> > Anyone got any ideas as to what the issue could be? > > > > > > > >>>> > > > > > > > > >>>> > Thank you, > > > > > > > >>>> > Clayton Turner > > > > > > > >>>> > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> -- > > > > > > > >>> -- > > > > > > > >>> Clayton Turner > > > > > > > >>> email: caturn...@g.cofc.edu > > > > > > > >>> phone: (843)-424-3784 > > > > > > > >>> web: claytonturner.blogspot.com > > > > > > > >>> > > > > > > > >>> > > > > > > > > > > > ------------------------------------------------------------------ > > > > ---- > > > > > > > --------------------------- > > > > > > > >>> “When scientifically investigating the natural world, > > > > > > > >>> the only thing worse than a blind believer is a seeing > > > > > > > >>> denier.” > > > > > > > >>> - Neil deGrasse Tyson > > > > > > > >>> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> -- > > > > > > > >> -- > > > > > > > >> Clayton Turner > > > > > > > >> email: caturn...@g.cofc.edu > > > > > > > >> phone: (843)-424-3784 > > > > > > > >> web: claytonturner.blogspot.com > > > > > > > >> > > > > > > > >> > > > > > > > > > > > ------------------------------------------------------------------ > > > > ---- > > > > > > > --------------------------- > > > > > > > >> “When scientifically investigating the natural world, the > > > > > > > >> only thing worse than a blind believer is a seeing denier.” > > > > > > > >> - Neil deGrasse Tyson > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > -- > > > > > > > > Clayton Turner > > > > > > > > email: caturn...@g.cofc.edu > > > > > > > > phone: (843)-424-3784 > > > > > > > > web: claytonturner.blogspot.com > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------ > > > > ---- > > > > > > > --------------------------- > > > > > > > > “When scientifically investigating the natural world, the > > > > > > > > only > > > > thing > > > > > > > worse > > > > > > > > than a blind believer is a seeing denier.” > > > > > > > > - Neil deGrasse Tyson > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > -- > > > > > > > Clayton Turner > > > > > > > email: caturn...@g.cofc.edu > > > > > > > phone: (843)-424-3784 > > > > > > > web: claytonturner.blogspot.com > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------ > > > > ---- > > > > > > > --------------------------- “When scientifically > > > > > > > investigating the natural world, the only thing worse than a > > > > > > > blind believer is a seeing denier.” > > > > > > > - Neil deGrasse Tyson > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > -- > > > Clayton Turner > > > email: caturn...@g.cofc.edu > > > phone: (843)-424-3784 > > > web: claytonturner.blogspot.com > > > > > ---------------------------------------------------------------------- > > --------------------------- > > > “When scientifically investigating the natural world, the only thing > > worse than a > > > blind believer is a seeing denier.” > > > - Neil deGrasse Tyson > > > > > > -- > -- > Clayton Turner > email: caturn...@g.cofc.edu > phone: (843)-424-3784 > web: claytonturner.blogspot.com > ------------------------------------------------------------------------------------------------- > “When scientifically investigating the natural world, the only thing worse > than a > blind believer is a seeing denier.” > - Neil deGrasse Tyson
