Re: [gpc-informatics] #202: Deploy Data Builder at each GPC site
#202: Deploy Data Builder at each GPC site --+--- Reporter: dconnolly | Owner: dconnolly Type: task | Status: assigned Priority: major | Milestone: bc-survey-cohort-def Component: data-sharing | Resolution: Keywords: breast-cancer-cohort | Blocked By: Blocking:| --+--- Comment (by finamore.joe): The URLs mentioned in comment 20 are correct. The situation is that we have moved past using the native PostgreSQL DB and are now pointing to our production SQL server database. We should be using the data_builder account created in the installation process, I presume, for this access instead of demo. When I look at the /var/log/data_builder/cdr2edc.log I see that it is trying to verify access to the i2b2 database for the demo account: 2015-01-16 10:14:28,615 INFO __main__ logging request parameters to: 2015-01-16T10:14:28.615923-demo.json 2015-01-16 10:14:28,616 INFO __main__ checking i2b2 password for: demo How can I move into production mode and specify the data_builder account instead for this access. Thanks. -- Ticket URL: http://informatics.gpcnetwork.org/trac/Project/ticket/202#comment:21 gpc-informatics http://informatics.gpcnetwork.org/ Greater Plains Network - Informatics ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev
Re: [gpc-informatics] #202: Deploy Data Builder at each GPC site
#202: Deploy Data Builder at each GPC site
--+---
Reporter: dconnolly | Owner: dconnolly
Type: task | Status: assigned
Priority: major | Milestone: bc-survey-cohort-def
Component: data-sharing | Resolution:
Keywords: breast-cancer-cohort | Blocked By:
Blocking:|
--+---
Comment (by dconnolly):
Replying to [comment:21 finamore.joe]:
... We should be using the data_builder account created in the
installation process, I presume, for this access instead of demo. When
I look at the /var/log/data_builder/cdr2edc.log I see that it is trying to
verify access to the i2b2 database for the demo account
The Error from back-end: incorrect credentials diagnostic indicates
failure to verify **i2b2 hive account** credentials with the i2b2 PM cell,
not database account credentials.
Does the `i2b2hive.py` integration test work?
{{{
Integration test usage::
$ python i2b2hive.py http:.../index.php http:.../PMService/ demo
demouser
}}}
--
Ticket URL:
http://informatics.gpcnetwork.org/trac/Project/ticket/202#comment:22
gpc-informatics http://informatics.gpcnetwork.org/
Greater Plains Network - Informatics
___
Gpc-dev mailing list
Gpc-dev@listserv.kumc.edu
http://listserv.kumc.edu/mailman/listinfo/gpc-dev
Re: [gpc-informatics] #205: interim builder for breast cancer cohort summary
#205: interim builder for breast cancer cohort summary --+--- Reporter: dconnolly | Owner: huhickman Type: enhancement | Status: assigned Priority: major | Milestone: bc-survey-cohort-def Component: data-sharing | Resolution: Keywords: breast-cancer-cohort | Blocked By: 32, 204 Blocking:| --+--- Comment (by bokov): Replying to [comment:2 dconnolly]: I just wrote up BuilderSaga#builder-sql-guide, based on a log of a data builder run. It includes the exact schema of the sqlite output file: attachment:builder_output_schema.sql:wiki:BuilderSaga as well as each SQL statement that the data builder runs. If in ours, the `variable` and `job` tables are blank, will that interfere with your being able to extract our data for the purposes of the breast cancer cohort selection? -- Ticket URL: http://informatics.gpcnetwork.org/trac/Project/ticket/205#comment:9 gpc-informatics http://informatics.gpcnetwork.org/ Greater Plains Network - Informatics ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev
remote participation details for HackathonTwo - test at 2pm
HackthonTwo remote participation detailshttps://informatics.gpcnetwork.org/trac/Project/wiki/HackathonTwo#remote are now in place. Copy for convenience: Remote Participation Use the attendance surveyhttps://redcap.uthscsa.edu/REDCap/surveys/?s=qcLVmeAfzt%22%3EHackathonTwo to register your interest in remote participation in a topic so we know which sessions will have remote participation and so we can help you get connected. The agenda schedulehttps://informatics.gpcnetwork.org/trac/Project/wiki/HackathonTwo#agenda calls for up to three sessions in parallel; we'll allocate these facilities to sessions as necessary: 1. GoToMeeting Meeting ID and Access Code: 476-096-581https://global.gotomeeting.com/join/476096581 * You can also dial in using your phone: +1 (872) 240-3212 or more phone numbershttps://global.gotomeeting.com/476096581/numbersdisplay.html 2. GoToMeeting Meeting ID and Access Code: 164-693-557https://global.gotomeeting.com/join/164693557 * You can also dial in using your phone: +1 (267) 507-0011 or more phone numbershttps://global.gotomeeting.com/164693557/numbersdisplay.html 3. HackathonTwo Google+ Eventhttps://plus.google.com/events/cmhvdu7s163si4tuhddbmep4hk0 with hangout for voice, video, chat Meanwhile if you want to catch up either in near-real-time or after the fact, see: * meeting notes shared dochttps://docs.google.com/document/d/13dA_ml1GSIhZ7fs-fWle5dPU95UlFQEoS6HJezSgKyQ/edit * see #12https://informatics.gpcnetwork.org/trac/Project/ticket/12 for meeting record norms Another possibly useful facility: * IRC channel: #gpc-devhttp://webchat.freenode.net?channels=%23gpc-devuio=d4 on freenodehttp://freenode.net/ * note public gpc-dev channel logshttps://botbot.me/freenode/gpc-dev/ * See #46https://informatics.gpcnetwork.org/trac/Project/ticket/46 about text chat Some of us are getting together at 2pm on the 1st gotomeeting (ID 476-096-581) to test the various facilities. You're welcome to join us. -- Dan ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev
NAACCR dates in i2b2
I'm wondering how other sites are representing dates from NAACCR in i2b2. At UMN all our observation_fact records from NAACCR are loaded with a start_date of the date_of_diagnosis. We are also loading facts that have concept_cd values like NAACCR:390|MMDD for all 3 date fields. I appreciate any input that can be provided. Thanks, -- Tim Meyer Programmer / Analyst Academic Health Center - Information Systems Phone: 612.624.8386 tme...@umn.edu ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev
Re: NAACCR dates in i2b2
Tim, thank you for the reminder on this — I knew that there was something I forgot to note for our data set. For the 4 date fields, we populated the UPDATE_DATE field in the OBSERVATION_FACT table to contain the referenced data from the underlying de-identified NAACCR source. This applied to: NAACCR|1750 NAACCR|390 NAACCR|1860 NAACCR|240 - Glenn Medical Informatics Senior Analyst CTSI – Clinical Translational Science Institute gbus...@mcw.edu (414) 805-7239 From: Tim Meyer tme...@umn.edumailto:tme...@umn.edu Date: Friday, January 16, 2015 at 1:18 PM To: gpc-dev@listserv.kumc.edumailto:gpc-dev@listserv.kumc.edu gpc-dev@listserv.kumc.edumailto:gpc-dev@listserv.kumc.edu Subject: NAACCR dates in i2b2 I'm wondering how other sites are representing dates from NAACCR in i2b2. At UMN all our observation_fact records from NAACCR are loaded with a start_date of the date_of_diagnosis. We are also loading facts that have concept_cd values like NAACCR:390|MMDD for all 3 date fields. I appreciate any input that can be provided. Thanks, -- Tim Meyer Programmer / Analyst Academic Health Center - Information Systems Phone: 612.624.8386 tme...@umn.edumailto:tme...@umn.edu ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev
RE: RxNorm metadata
I am getting closer, the KU and the UTSW metadata on Bable both have modifiers, the GPC does not. I what do I need to do to end up with KUMC style modifiers? Michael Munns Database Analyst 402-559-3821 From: Nathan Graham [mailto:ngra...@kumc.edu] Sent: Monday, December 08, 2014 10:45 AM To: Munns, Michael B; gpc-dev@listserv.kumc.edu Subject: RE: RxNorm metadata We used the exclude modifiers table to remove the modifiers from things that didn't really need them like NO HOME MEDICATIONS, EMPTY CONTAINER, PLASTIC BAG, MISCELLANEOUS MEDICAL SUPPLY MISC, etc. These are things we put directly under the top level Medications folder as we didn't know where else to put them. Maybe it could be argued that some of these don't have much research value - I don't know. But, many of these concepts had lots of patient/fact counts so we didn't want to just exclude them. At least the NO HOME MEDICATIONS item was specifically discussed and deemed worth addinghttps://informatics.kumc.edu/work/ticket/1775. Our (KUMC) general approach is bring in everything - that way, we can all can see what we have available. If/when we find that things can be improved (based on user feedback, etc) we can do that later. From: gpc-dev-boun...@listserv.kumc.edumailto:gpc-dev-boun...@listserv.kumc.edu [mailto:gpc-dev-boun...@listserv.kumc.edu] On Behalf Of Munns, Michael B Sent: Monday, December 08, 2014 10:30 AM To: gpc-dev@listserv.kumc.edumailto:gpc-dev@listserv.kumc.edu Subject: FW: RxNorm metadata We can put them on the mailing list. We did end up creating an empty table for the manual curation, also a empty table for the exclude modifiers. Michael Munns Database Analyst 402-559-3821 From: Nathan Graham [mailto:ngra...@kumc.edu] Sent: Monday, December 08, 2014 10:18 AM To: Munns, Michael B Subject: RE: RxNorm metadata Michael, I'd really like to keep these conversations on the gpc-devhttp://listserv.kumc.edu/pipermail/gpc-dev/ mailing list (sign up from herehttp://listserv.kumc.edu/mailman/listinfo/gpc-dev). If you have questions I think others will too. And, I think we all benefit from as much collaboration as we can have. Could we move future questions there? I think what you want is an empty table right now. The manual curation table is what it says - a place for you to manually specify parents for medications that we didn't link from Epic to RxNorm. The documentation could probably be better - from the comment at the top of the file: Notes about manual curation: med_map_manual_curation is a table that maps a clarity medication ID to a parent concept. It is designed to be used for medications that we couldn't map to RxNORM directly from information in Clarity (via GCN, NDC, etc). This parent could be a VA class or an SCDF/SBDF. The table has the following columns: CLARITY_MEDICATION_ID - the id of the med without direct linkage to RxNORM CLARITY_NAME (not used for mapping - just for eyeballing) FACTS (not used for mapping - just for eyeballing) PATIENTS (not used for mapping - just for eyeballing) VA_NAME (not used for mapping - just for eyeballing) VA_RXAUI - If we didn't map to an SDF AUI (below), the fall back on this one for VA class. SDF_RXAUI - Map here if not null, otherwise fall back on the VA_RXAUI (above) SDF_NAME (not used for mapping - just for eyeballing) I know I've probably made KU-specific assumptions in the code that make porting difficult. So, if you have improvements, suggestions, etc I'd be very happy to look at patches. Or, if you start using the Elephant repository (see MultiSiteDevhttps://informatics.gpcnetwork.org/trac/Project/wiki/MultiSiteDev) you could commit directly to a branch and we could work together to merge your changes into default as appropriate. Thanks. Regards, Nathan From: Munns, Michael B [mailto:mike.mu...@unmc.edu] Sent: Thursday, December 04, 2014 4:18 PM To: Nathan Graham Subject: RE: RxNorm metadata Importance: High I have more questions about this rxnorm_terms table. On line 274 there is a create the I am running as an insert, that contains a union for the manual curation. If I am following all of this correct, and I am not sure that I am, some of this may be that it's the first time we've ran and the rxnorm_terms is empty. To start though there is a view mapped_meds , that uses rxnorm_terms. Then the view unmapped_meds is built off of mapped_meds. The unmapped_meds view is used in a query to create the med_map_manual_curation.csv which is used to Populate the med_map_manual_curation table. The union in the create @274 calls the med_map_manual_curation table , which does not exist. The join is where its med id is not null. So I could remove that section of the code or put an empty table out there. Any downside to an empty table or skipping that union you are aware of? Am I missing something? Michael Munns Database Analyst 402-559-3821 From: Munns, Michael B Sent: Thursday, December 04, 2014 1:41 PM To: 'Nathan
RE: NAACCR dates in i2b2
See also Nov 7 discussion. http://listserv.kumc.edu/pipermail/gpc-dev/2014q4/000778.html -- Dan From: gpc-dev-boun...@listserv.kumc.edu [gpc-dev-boun...@listserv.kumc.edu] on behalf of Bushee, Glenn [gbus...@mcw.edu] Sent: Friday, January 16, 2015 1:29 PM To: Tim Meyer; gpc-dev@listserv.kumc.edu Subject: Re: NAACCR dates in i2b2 Tim, thank you for the reminder on this — I knew that there was something I forgot to note for our data set. For the 4 date fields, we populated the UPDATE_DATE field in the OBSERVATION_FACT table to contain the referenced data from the underlying de-identified NAACCR source. This applied to: NAACCR|1750 NAACCR|390 NAACCR|1860 NAACCR|240 - Glenn Medical Informatics Senior Analyst CTSI – Clinical Translational Science Institute gbus...@mcw.edu (414) 805-7239 From: Tim Meyer tme...@umn.edumailto:tme...@umn.edu Date: Friday, January 16, 2015 at 1:18 PM To: gpc-dev@listserv.kumc.edumailto:gpc-dev@listserv.kumc.edu gpc-dev@listserv.kumc.edumailto:gpc-dev@listserv.kumc.edu Subject: NAACCR dates in i2b2 I'm wondering how other sites are representing dates from NAACCR in i2b2. At UMN all our observation_fact records from NAACCR are loaded with a start_date of the date_of_diagnosis. We are also loading facts that have concept_cd values like NAACCR:390|MMDD for all 3 date fields. I appreciate any input that can be provided. Thanks, -- Tim Meyer Programmer / Analyst Academic Health Center - Information Systems Phone: 612.624.8386 tme...@umn.edumailto:tme...@umn.edu ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev
Re: remote participation details for HackathonTwo - test at 2pm
UMN folks, if you have a few minutes, perhaps you'd like to join the below conference to confirm that everything will go smoothly end-to-end during the real Hackathon? Our away team got a little held up heading over to the future site of the Hackathon, but the test should begin in about 10 minutes. On 01/16/2015 12:15 PM, Dan Connolly wrote: HackthonTwo remote participation details https://informatics.gpcnetwork.org/trac/Project/wiki/HackathonTwo#remote are now in place. Copy for convenience: Remote Participation *Use the attendance survey https://redcap.uthscsa.edu/REDCap/surveys/?s=qcLVmeAfzt%22%3EHackathonTwo to register your interest in remote participation in a topic so we know which sessions will have remote participation and so we can help you get connected*. The agenda schedule https://informatics.gpcnetwork.org/trac/Project/wiki/HackathonTwo#agenda calls for up to three sessions in parallel; we'll allocate these facilities to sessions as necessary: 1. GoToMeeting Meeting ID and Access Code: 476-096-581 https://global.gotomeeting.com/join/476096581 * You can also dial in using your phone: +1 (872) 240-3212 or more phone numbers https://global.gotomeeting.com/476096581/numbersdisplay.html 2. GoToMeeting Meeting ID and Access Code: 164-693-557 https://global.gotomeeting.com/join/164693557 * You can also dial in using your phone: +1 (267) 507-0011 or more phone numbers https://global.gotomeeting.com/164693557/numbersdisplay.html 3. HackathonTwo Google+ Event https://plus.google.com/events/cmhvdu7s163si4tuhddbmep4hk0 with hangout for voice, video, chat Meanwhile if you want to catch up either in near-real-time or after the fact, see: * meeting notes shared doc https://docs.google.com/document/d/13dA_ml1GSIhZ7fs-fWle5dPU95UlFQEoS6HJezSgKyQ/edit o see #12 https://informatics.gpcnetwork.org/trac/Project/ticket/12 for meeting record norms Another possibly useful facility: * IRC channel: #gpc-dev http://webchat.freenode.net?channels=%23gpc-devuio=d4 on freenode http://freenode.net/ o note public gpc-dev channel logs https://botbot.me/freenode/gpc-dev/ o See #46 https://informatics.gpcnetwork.org/trac/Project/ticket/46 about text chat Some of us are getting together at 2pm on the 1st gotomeeting (ID 476-096-581) to test the various facilities. You're welcome to join us. ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev
Re: [gpc-informatics] #202: Deploy Data Builder at each GPC site
#202: Deploy Data Builder at each GPC site --+--- Reporter: dconnolly | Owner: dconnolly Type: task | Status: assigned Priority: major | Milestone: bc-survey-cohort-def Component: data-sharing | Resolution: Keywords: breast-cancer-cohort | Blocked By: Blocking:| --+--- Comment (by finamore.joe): I apologize to the group for spamming them, but it is important that I get this working. I executed the i2b2hive.py integration test and have attached the results to this. If anyone would be inclined to work with me directly, that would be fine (finamore@mcrf.mfldclin.edu). As near as I can tell, it got past the 'checked_access()' call but am not 100% sure. Thank you. -- Ticket URL: http://informatics.gpcnetwork.org/trac/Project/ticket/202#comment:23 gpc-informatics http://informatics.gpcnetwork.org/ Greater Plains Network - Informatics ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev
Re: NAACCR dates in i2b2
Thanks. This is what I was looking for. Tim On Fri, Jan 16, 2015 at 2:11 PM, Dan Connolly dconno...@kumc.edu wrote: See also Nov 7 discussion. http://listserv.kumc.edu/pipermail/gpc-dev/2014q4/000778.html -- Dan From: gpc-dev-boun...@listserv.kumc.edu [gpc-dev-boun...@listserv.kumc.edu] on behalf of Bushee, Glenn [gbus...@mcw.edu] Sent: Friday, January 16, 2015 1:29 PM To: Tim Meyer; gpc-dev@listserv.kumc.edu Subject: Re: NAACCR dates in i2b2 Tim, thank you for the reminder on this — I knew that there was something I forgot to note for our data set. For the 4 date fields, we populated the UPDATE_DATE field in the OBSERVATION_FACT table to contain the referenced data from the underlying de-identified NAACCR source. This applied to: NAACCR|1750 NAACCR|390 NAACCR|1860 NAACCR|240 - Glenn Medical Informatics Senior Analyst CTSI – Clinical Translational Science Institute gbus...@mcw.edu (414) 805-7239 From: Tim Meyer tme...@umn.edumailto:tme...@umn.edu Date: Friday, January 16, 2015 at 1:18 PM To: gpc-dev@listserv.kumc.edumailto:gpc-dev@listserv.kumc.edu gpc-dev@listserv.kumc.edumailto:gpc-dev@listserv.kumc.edu Subject: NAACCR dates in i2b2 I'm wondering how other sites are representing dates from NAACCR in i2b2. At UMN all our observation_fact records from NAACCR are loaded with a start_date of the date_of_diagnosis. We are also loading facts that have concept_cd values like NAACCR:390|MMDD for all 3 date fields. I appreciate any input that can be provided. Thanks, -- Tim Meyer Programmer / Analyst Academic Health Center - Information Systems Phone: 612.624.8386 tme...@umn.edumailto:tme...@umn.edu ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev -- Tim Meyer Programmer / Analyst Academic Health Center - Information Systems Phone: 612.624.8386 tme...@umn.edu ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev
Re: [gpc-informatics] #205: interim builder for breast cancer cohort summary
#205: interim builder for breast cancer cohort summary --+--- Reporter: dconnolly | Owner: huhickman Type: enhancement | Status: assigned Priority: major | Milestone: bc-survey-cohort-def Component: data-sharing | Resolution: Keywords: breast-cancer-cohort | Blocked By: 32, 204 Blocking:| --+--- Comment (by bos): Replying to [comment:10 dconnolly]: Replying to [comment:9 bokov]: If in ours, the `variable` and `job` tables are blank, will that interfere with your being able to extract our data for the purposes of the breast cancer cohort selection? I expect we can do without `job`, but we rely in `variable`. It's computable from the XML definition of your query. By design, everybody is using **exactly** the same query (#204), so we can just copy it from another data file. But it's reasonably important to be sure. If you do a timeline of your results, you can use `timeline_to_job.py` (from the databuilder code) to compute a JSON file with the relevant details. With a little tweaking to not look for a patient set id, it would probably work on an XML query definition. Then see `export_terms()` in `dfbuilder.py` for how we get the data into the variable table. It might be easier to just write your own 50 line script; or perhaps I could factor the relevant parts out. I think the relationship between the query definition and the variable table is pretty straightforward. If necessary, just give me the XML query definition that you ran and I'll compute it. [[BR]] We had some i2b2 issues, so we had to build run the query manually, then convert it to sqlite. Hence, hence our job and variable tables are empty. -- Ticket URL: http://informatics.gpcnetwork.org/trac/Project/ticket/205#comment:11 gpc-informatics http://informatics.gpcnetwork.org/ Greater Plains Network - Informatics ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev
Re: [gpc-informatics] #202: Deploy Data Builder at a critical mass of GPC sites (was: Deploy Data Builder at each GPC site)
#202: Deploy Data Builder at a critical mass of GPC sites --+--- Reporter: dconnolly | Owner: dconnolly Type: task | Status: closed Priority: major | Milestone: bc-survey-cohort-def Component: data-sharing | Resolution: fixed Keywords: breast-cancer-cohort | Blocked By: Blocking:| --+--- Changes (by dconnolly): * status: assigned = closed * resolution: = fixed Comment: Done: KUMC, WISC, UNMC, UTHSCSA, UTSW tracked as separate tickets: [[TicketQuery(id=217|218|168|205)]] -- Ticket URL: http://informatics.gpcnetwork.org/trac/Project/ticket/202#comment:24 gpc-informatics http://informatics.gpcnetwork.org/ Greater Plains Network - Informatics ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev
Re: [gpc-informatics] #205: interim builder for breast cancer cohort summary at MCW, UMN (was: interim builder for breast cancer cohort summary)
#205: interim builder for breast cancer cohort summary at MCW, UMN --+--- Reporter: dconnolly | Owner: huhickman Type: enhancement | Status: assigned Priority: major | Milestone: bc-survey-cohort-def Component: data-sharing | Resolution: Keywords: breast-cancer-cohort | Blocked By: 32, 204 Blocking:| --+--- Comment (by dconnolly): re-scoping this a bit while closing #202. -- Ticket URL: http://informatics.gpcnetwork.org/trac/Project/ticket/205#comment:12 gpc-informatics http://informatics.gpcnetwork.org/ Greater Plains Network - Informatics ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev
