Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
Dave, It's an oversight that we didn't document that. Thanks for pointing that out! I'll update the documentation tomorrow and post again with proper details of the command to use. The short answer is that hsqldb jar includes a GUI. You use the 2.3.4 version of that jar, which is included with the cTAKES 4.0 convenience binary, to open your database. Then you close (shutdown) your DB and it gets converted to 2.3.4 for you. But if the .properties file for your DB indicates it's readonly, you need to have edited the properties file first to remove that. That gives you the idea. I'll write it up properly tomorrow. Thanks for testing, -- James *Sent from my phone.* On Apr 22, 2017 9:39 PM, "David Kincaid" wrote: I finally had some time this weekend to give 4.0.0rc3 a run. The install and configuration instructions worked perfectly on my Linux laptop and I was able to use CVD to run the clinical pipeline successfully against some of the clinical notes I have here. However, I ran into a problem when I tried running my own customized pipeline that includes a custom dictionary that I create (to include the SNOMED veterinary extension). I get an exception while it's loading that the dictionary is not the correct version. So, I assume I just need to recreate that custom database? Or is there a migration utility from the old database version to the new version? I don't remember seeing mention of this, but certainly could have missed it. If it's not in the release notes or install/upgrade instructions it probably should be there. Thanks to everyone for pulling this release together and getting it out the door! - Dave
RE: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
Thank you James - great explanation and information. Hi David, There is a new version of the dictionary creator gui distributed with ctakes 4.0. It is 99% the same as the version in sandbox. One important difference is that it now produces a database in hsqldb 2.3.4 - compatible with ctakes 4.0. Another difference (important to you) is that there is an exclusion list in a data file that lists cui / term combinations that can be excluded. By default the medications "toDAY" and "ToMORROW" are in that list. It is obviously done to prevent the frequent false positives that you spoke of in your Amia presentation. For non-vet use these exclusions are pretty valid. Since you are interested in a vet dictionary fi you use the new dictionary creator you can decide whether or not you want them to be included. Sean -Original Message- From: James Masanz [mailto:masanz.ja...@gmail.com] Sent: Saturday, April 22, 2017 11:32 PM To: dev@ctakes.apache.org Subject: Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding? Dave, It's an oversight that we didn't document that. Thanks for pointing that out! I'll update the documentation tomorrow and post again with proper details of the command to use. The short answer is that hsqldb jar includes a GUI. You use the 2.3.4 version of that jar, which is included with the cTAKES 4.0 convenience binary, to open your database. Then you close (shutdown) your DB and it gets converted to 2.3.4 for you. But if the .properties file for your DB indicates it's readonly, you need to have edited the properties file first to remove that. That gives you the idea. I'll write it up properly tomorrow. Thanks for testing, -- James *Sent from my phone.* On Apr 22, 2017 9:39 PM, "David Kincaid" wrote: I finally had some time this weekend to give 4.0.0rc3 a run. The install and configuration instructions worked perfectly on my Linux laptop and I was able to use CVD to run the clinical pipeline successfully against some of the clinical notes I have here. However, I ran into a problem when I tried running my own customized pipeline that includes a custom dictionary that I create (to include the SNOMED veterinary extension). I get an exception while it's loading that the dictionary is not the correct version. So, I assume I just need to recreate that custom database? Or is there a migration utility from the old database version to the new version? I don't remember seeing mention of this, but certainly could have missed it. If it's not in the release notes or install/upgrade instructions it probably should be there. Thanks to everyone for pulling this release together and getting it out the door! - Dave
Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
Sorry about this. Actually no, it's not working. I just created a new blank database apparently in a new directory. Still having the original problem trying to open the DB in HSQL GUI. Any ideas? - Dave On Sun, Apr 23, 2017 at 2:23 PM, David Kincaid wrote: > I got it figured out. I was using the wrong URL path for the database. > Working great now! > > - Dave > > On Sun, Apr 23, 2017 at 12:31 PM, David Kincaid > wrote: > >> Thanks for the reply, James. I'm not very familiar with HSQLDB, so I may >> be doing it wrong, but I launched the GUI using "java >> -jar hsqldb-2.3.4.jar" and then tried to connect to the DB by changing the >> "Type" to "HSQL Database Engine Standalone" and then setting the URL to >> point to my custom dictionary directory. It throws a SQLException that says >> "wrong database file version". Is there some other step I'm missing? >> >> - Dave >> >> On Sat, Apr 22, 2017 at 10:31 PM, James Masanz >> wrote: >> >>> Dave, >>> >>> It's an oversight that we didn't document that. Thanks for pointing that >>> out! I'll update the documentation tomorrow and post again with proper >>> details of the command to use. >>> >>> The short answer is that hsqldb jar includes a GUI. You use the 2.3.4 >>> version of that jar, which is included with the cTAKES 4.0 convenience >>> binary, to open your database. Then you close (shutdown) your DB and it >>> gets converted to 2.3.4 for you. But if the .properties file for your DB >>> indicates it's readonly, you need to have edited the properties file >>> first >>> to remove that. That gives you the idea. I'll write it up properly >>> tomorrow. >>> >>> Thanks for testing, >>> -- James >>> >>> >>> *Sent from my phone.* >>> >>> On Apr 22, 2017 9:39 PM, "David Kincaid" wrote: >>> >>> I finally had some time this weekend to give 4.0.0rc3 a run. The install >>> and configuration instructions worked perfectly on my Linux laptop and I >>> was able to use CVD to run the clinical pipeline successfully against >>> some >>> of the clinical notes I have here. >>> >>> However, I ran into a problem when I tried running my own customized >>> pipeline that includes a custom dictionary that I create (to include the >>> SNOMED veterinary extension). I get an exception while it's loading that >>> the dictionary is not the correct version. So, I assume I just need to >>> recreate that custom database? Or is there a migration utility from the >>> old >>> database version to the new version? >>> >>> I don't remember seeing mention of this, but certainly could have missed >>> it. If it's not in the release notes or install/upgrade instructions it >>> probably should be there. >>> >>> Thanks to everyone for pulling this release together and getting it out >>> the >>> door! >>> >>> - Dave >>> >> >> >
Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
I've added a section[1] to the User Install Guide describing how to convert an HSQLDB from hsqldb 1.8 to 2.3.4 for cTAKES 4.0 [1] https://cwiki.apache.org/confluence/display/CTAKES/cTAKES+4.0+User+Install+Guide#cTAKES4.0UserInstallGuide-ConvertDictionariesYou'vePreviouslyCreatedtobeCompatiblewithcTAKES4.0 On Sun, Apr 23, 2017 at 10:03 AM, Finan, Sean < sean.fi...@childrens.harvard.edu> wrote: > Thank you James - great explanation and information. > > Hi David, > There is a new version of the dictionary creator gui distributed with > ctakes 4.0. It is 99% the same as the version in sandbox. One important > difference is that it now produces a database in hsqldb 2.3.4 - compatible > with ctakes 4.0. > Another difference (important to you) is that there is an exclusion list > in a data file that lists cui / term combinations that can be excluded. By > default the medications "toDAY" and "ToMORROW" are in that list. It is > obviously done to prevent the frequent false positives that you spoke of in > your Amia presentation. For non-vet use these exclusions are pretty > valid. Since you are interested in a vet dictionary fi you use the new > dictionary creator you can decide whether or not you want them to be > included. > > Sean > > -Original Message- > From: James Masanz [mailto:masanz.ja...@gmail.com] > Sent: Saturday, April 22, 2017 11:32 PM > To: dev@ctakes.apache.org > Subject: Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding? > > Dave, > > It's an oversight that we didn't document that. Thanks for pointing that > out! I'll update the documentation tomorrow and post again with proper > details of the command to use. > > The short answer is that hsqldb jar includes a GUI. You use the 2.3.4 > version of that jar, which is included with the cTAKES 4.0 convenience > binary, to open your database. Then you close (shutdown) your DB and it > gets converted to 2.3.4 for you. But if the .properties file for your DB > indicates it's readonly, you need to have edited the properties file first > to remove that. That gives you the idea. I'll write it up properly tomorrow. > > Thanks for testing, > -- James > > > *Sent from my phone.* > > On Apr 22, 2017 9:39 PM, "David Kincaid" wrote: > > I finally had some time this weekend to give 4.0.0rc3 a run. The install > and configuration instructions worked perfectly on my Linux laptop and I > was able to use CVD to run the clinical pipeline successfully against some > of the clinical notes I have here. > > However, I ran into a problem when I tried running my own customized > pipeline that includes a custom dictionary that I create (to include the > SNOMED veterinary extension). I get an exception while it's loading that > the dictionary is not the correct version. So, I assume I just need to > recreate that custom database? Or is there a migration utility from the old > database version to the new version? > > I don't remember seeing mention of this, but certainly could have missed > it. If it's not in the release notes or install/upgrade instructions it > probably should be there. > > Thanks to everyone for pulling this release together and getting it out > the door! > > - Dave >
Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
Thanks for the reply, James. I'm not very familiar with HSQLDB, so I may be doing it wrong, but I launched the GUI using "java -jar hsqldb-2.3.4.jar" and then tried to connect to the DB by changing the "Type" to "HSQL Database Engine Standalone" and then setting the URL to point to my custom dictionary directory. It throws a SQLException that says "wrong database file version". Is there some other step I'm missing? - Dave On Sat, Apr 22, 2017 at 10:31 PM, James Masanz wrote: > Dave, > > It's an oversight that we didn't document that. Thanks for pointing that > out! I'll update the documentation tomorrow and post again with proper > details of the command to use. > > The short answer is that hsqldb jar includes a GUI. You use the 2.3.4 > version of that jar, which is included with the cTAKES 4.0 convenience > binary, to open your database. Then you close (shutdown) your DB and it > gets converted to 2.3.4 for you. But if the .properties file for your DB > indicates it's readonly, you need to have edited the properties file first > to remove that. That gives you the idea. I'll write it up properly > tomorrow. > > Thanks for testing, > -- James > > > *Sent from my phone.* > > On Apr 22, 2017 9:39 PM, "David Kincaid" wrote: > > I finally had some time this weekend to give 4.0.0rc3 a run. The install > and configuration instructions worked perfectly on my Linux laptop and I > was able to use CVD to run the clinical pipeline successfully against some > of the clinical notes I have here. > > However, I ran into a problem when I tried running my own customized > pipeline that includes a custom dictionary that I create (to include the > SNOMED veterinary extension). I get an exception while it's loading that > the dictionary is not the correct version. So, I assume I just need to > recreate that custom database? Or is there a migration utility from the old > database version to the new version? > > I don't remember seeing mention of this, but certainly could have missed > it. If it's not in the release notes or install/upgrade instructions it > probably should be there. > > Thanks to everyone for pulling this release together and getting it out the > door! > > - Dave >
Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
I got it figured out. I was using the wrong URL path for the database. Working great now! - Dave On Sun, Apr 23, 2017 at 12:31 PM, David Kincaid wrote: > Thanks for the reply, James. I'm not very familiar with HSQLDB, so I may > be doing it wrong, but I launched the GUI using "java > -jar hsqldb-2.3.4.jar" and then tried to connect to the DB by changing the > "Type" to "HSQL Database Engine Standalone" and then setting the URL to > point to my custom dictionary directory. It throws a SQLException that says > "wrong database file version". Is there some other step I'm missing? > > - Dave > > On Sat, Apr 22, 2017 at 10:31 PM, James Masanz > wrote: > >> Dave, >> >> It's an oversight that we didn't document that. Thanks for pointing that >> out! I'll update the documentation tomorrow and post again with proper >> details of the command to use. >> >> The short answer is that hsqldb jar includes a GUI. You use the 2.3.4 >> version of that jar, which is included with the cTAKES 4.0 convenience >> binary, to open your database. Then you close (shutdown) your DB and it >> gets converted to 2.3.4 for you. But if the .properties file for your DB >> indicates it's readonly, you need to have edited the properties file first >> to remove that. That gives you the idea. I'll write it up properly >> tomorrow. >> >> Thanks for testing, >> -- James >> >> >> *Sent from my phone.* >> >> On Apr 22, 2017 9:39 PM, "David Kincaid" wrote: >> >> I finally had some time this weekend to give 4.0.0rc3 a run. The install >> and configuration instructions worked perfectly on my Linux laptop and I >> was able to use CVD to run the clinical pipeline successfully against some >> of the clinical notes I have here. >> >> However, I ran into a problem when I tried running my own customized >> pipeline that includes a custom dictionary that I create (to include the >> SNOMED veterinary extension). I get an exception while it's loading that >> the dictionary is not the correct version. So, I assume I just need to >> recreate that custom database? Or is there a migration utility from the >> old >> database version to the new version? >> >> I don't remember seeing mention of this, but certainly could have missed >> it. If it's not in the release notes or install/upgrade instructions it >> probably should be there. >> >> Thanks to everyone for pulling this release together and getting it out >> the >> door! >> >> - Dave >> > >
Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
Dave, Is there something we can clarify on how to create the URL? On Sun, Apr 23, 2017 at 3:23 PM, David Kincaid wrote: > I got it figured out. I was using the wrong URL path for the database. > Working great now! > > - Dave > > On Sun, Apr 23, 2017 at 12:31 PM, David Kincaid > wrote: > > > Thanks for the reply, James. I'm not very familiar with HSQLDB, so I may > > be doing it wrong, but I launched the GUI using "java > > -jar hsqldb-2.3.4.jar" and then tried to connect to the DB by changing > the > > "Type" to "HSQL Database Engine Standalone" and then setting the URL to > > point to my custom dictionary directory. It throws a SQLException that > says > > "wrong database file version". Is there some other step I'm missing? > > > > - Dave > > > > On Sat, Apr 22, 2017 at 10:31 PM, James Masanz > > wrote: > > > >> Dave, > >> > >> It's an oversight that we didn't document that. Thanks for pointing that > >> out! I'll update the documentation tomorrow and post again with proper > >> details of the command to use. > >> > >> The short answer is that hsqldb jar includes a GUI. You use the 2.3.4 > >> version of that jar, which is included with the cTAKES 4.0 convenience > >> binary, to open your database. Then you close (shutdown) your DB and it > >> gets converted to 2.3.4 for you. But if the .properties file for your DB > >> indicates it's readonly, you need to have edited the properties file > first > >> to remove that. That gives you the idea. I'll write it up properly > >> tomorrow. > >> > >> Thanks for testing, > >> -- James > >> > >> > >> *Sent from my phone.* > >> > >> On Apr 22, 2017 9:39 PM, "David Kincaid" > wrote: > >> > >> I finally had some time this weekend to give 4.0.0rc3 a run. The install > >> and configuration instructions worked perfectly on my Linux laptop and I > >> was able to use CVD to run the clinical pipeline successfully against > some > >> of the clinical notes I have here. > >> > >> However, I ran into a problem when I tried running my own customized > >> pipeline that includes a custom dictionary that I create (to include the > >> SNOMED veterinary extension). I get an exception while it's loading that > >> the dictionary is not the correct version. So, I assume I just need to > >> recreate that custom database? Or is there a migration utility from the > >> old > >> database version to the new version? > >> > >> I don't remember seeing mention of this, but certainly could have missed > >> it. If it's not in the release notes or install/upgrade instructions it > >> probably should be there. > >> > >> Thanks to everyone for pulling this release together and getting it out > >> the > >> door! > >> > >> - Dave > >> > > > > >
Re: cTAKES 4.0rc3 - custom dictionary needs rebuilding?
I ended up just using the DictionaryCreator and recreating the dictionary, so it's working now. I haven't looked at the section you added to the user guide yet, but should have some time tonight to give it a go and test it. Thanks, Dave On Mon, Apr 24, 2017 at 10:29 AM, James Masanz wrote: > Dave, > Is there something we can clarify on how to create the URL? > > On Sun, Apr 23, 2017 at 3:23 PM, David Kincaid > wrote: > > > I got it figured out. I was using the wrong URL path for the database. > > Working great now! > > > > - Dave > > > > On Sun, Apr 23, 2017 at 12:31 PM, David Kincaid > > wrote: > > > > > Thanks for the reply, James. I'm not very familiar with HSQLDB, so I > may > > > be doing it wrong, but I launched the GUI using "java > > > -jar hsqldb-2.3.4.jar" and then tried to connect to the DB by changing > > the > > > "Type" to "HSQL Database Engine Standalone" and then setting the URL to > > > point to my custom dictionary directory. It throws a SQLException that > > says > > > "wrong database file version". Is there some other step I'm missing? > > > > > > - Dave > > > > > > On Sat, Apr 22, 2017 at 10:31 PM, James Masanz > > > > wrote: > > > > > >> Dave, > > >> > > >> It's an oversight that we didn't document that. Thanks for pointing > that > > >> out! I'll update the documentation tomorrow and post again with > proper > > >> details of the command to use. > > >> > > >> The short answer is that hsqldb jar includes a GUI. You use the 2.3.4 > > >> version of that jar, which is included with the cTAKES 4.0 convenience > > >> binary, to open your database. Then you close (shutdown) your DB and > it > > >> gets converted to 2.3.4 for you. But if the .properties file for your > DB > > >> indicates it's readonly, you need to have edited the properties file > > first > > >> to remove that. That gives you the idea. I'll write it up properly > > >> tomorrow. > > >> > > >> Thanks for testing, > > >> -- James > > >> > > >> > > >> *Sent from my phone.* > > >> > > >> On Apr 22, 2017 9:39 PM, "David Kincaid" > > wrote: > > >> > > >> I finally had some time this weekend to give 4.0.0rc3 a run. The > install > > >> and configuration instructions worked perfectly on my Linux laptop > and I > > >> was able to use CVD to run the clinical pipeline successfully against > > some > > >> of the clinical notes I have here. > > >> > > >> However, I ran into a problem when I tried running my own customized > > >> pipeline that includes a custom dictionary that I create (to include > the > > >> SNOMED veterinary extension). I get an exception while it's loading > that > > >> the dictionary is not the correct version. So, I assume I just need to > > >> recreate that custom database? Or is there a migration utility from > the > > >> old > > >> database version to the new version? > > >> > > >> I don't remember seeing mention of this, but certainly could have > missed > > >> it. If it's not in the release notes or install/upgrade instructions > it > > >> probably should be there. > > >> > > >> Thanks to everyone for pulling this release together and getting it > out > > >> the > > >> door! > > >> > > >> - Dave > > >> > > > > > > > > >