RE: how to preserve saved queries on i2b2 refresh?
Yes, we asked, and this is a very helpful write up and set of links. Thanks! -Angela From: gpc-dev-boun...@listserv.kumc.edu [mailto:gpc-dev-boun...@listserv.kumc.edu] On Behalf Of Dan Connolly Sent: Thursday, December 04, 2014 4:58 PM To: Wanta Keith M; gpc-dev@listserv.kumc.edu Subject: RE: how to preserve saved queries on i2b2 refresh? The short answer, for HERON at KUMC, is: * Saved queries are supported indefinitely. * We mostly try to evolve terminologies gracefully, * but sometimes we don't bother and we let queries with those terms break. * Saved patient sets are supported for 3 months. To elaborate on the honest broker and version control conversation... i2b2 stores both queries and patient sets. How to preserve them across refreshes? Essentially, back up and restore the relevant tables and schemas (PM, QT, workplace). For (some) details, see HeronLoad#Productioncut-overhttps://informatics.kumc.edu/work/wiki/HeronLoad#Productioncut-over. Do patient ids change? yes; we regenerate them every month, which invalidates any saved patient sets. This was rude to users who would build a patient set on the 30th, only to find it's horked the next day. Our approach to this honest broker issue is: we update patient sets, filling in the corresponding new patient_nums, but only for three months; if we did it indefinitely, then the patient_num could serve as a long-term patient identifier, largely defeating of its privacy protections. (for reference: ticket #699, July 2013 sappa releasehttps://informatics.kumc.edu/work/blog/heron-sappa-update) And yes, there's a lot to talk about w.r.t. ontology version control; we have a dozen or so tickets tagged terminology-evolutionhttps://informatics.kumc.edu/work/query?keywords=%7Eterminology-evolutioncol=idcol=summarycol=keywordscol=statuscol=ownercol=typecol=priorityorder=priority (only about half published :-/). If a user re-runs a query and the terminology has changed so that some of the query's terms are no longer there, it fails (with no clue to the user what's going on, either :-/). So in cases such as #1955https://informatics.kumc.edu/work/ticket/1955 where our notes tree had a clumsy extra ROOT node... Choices include: * Get rid of it * Let any existing queries that use terms below it in the hierarchy break. * Preserve old queries by maintaining a hidden tree (as for old diagnosis terms in #441 and source:heron_load/concepts_merge.sql#L278https://informatics.kumc.edu/work/browser/heron_load/concepts_merge.sql#L278 ) * Keep it In that case, ... we decided not to worry about preserving old queries, whereas when we refined our diagnosis terms, we did keep the old terms, but with c_visualattributes set to hide them. p.s. I didn't ask the question; I think the question was from Angela or Alex in San Antonio. I just brought it here for discussion from the meeting notes. -- Dan From: Wanta Keith M [kwa...@uwhealth.org] Sent: Tuesday, December 02, 2014 6:27 PM To: Dan Connolly; gpc-dev@listserv.kumc.edumailto:gpc-dev@listserv.kumc.edu Subject: RE: how to preserve saved queries on i2b2 refresh? Very good question Dan! UW Madison currently disables old queries on each i2b2 refresh if our concepts change. So we just do it for each release for now to be safe. We are not handling patient ids (or any ids) correctly yet after a refresh. This is an ice breaker for a great honest broker and ontology version control conversation. From: gpc-dev-boun...@listserv.kumc.edumailto:gpc-dev-boun...@listserv.kumc.edu [mailto:gpc-dev-boun...@listserv.kumc.edu] On Behalf Of Dan Connolly Sent: Tuesday, December 02, 2014 1:48 PM To: gpc-dev@listserv.kumc.edumailto:gpc-dev@listserv.kumc.edu Subject: how to preserve saved queries on i2b2 refresh? I didn't notice this question in today's meeting notes: * Question: on re-deploy to i2b2, how to preserve saved queries? Do patient IDs, etc change. Does i2b2 re-run saved queries or use stored patient sets with old ids? Here's hoping for time to answer... or maybe Nathan will beat me to it. grumble. We have extensive internal documentation on this issue; I wish I could just point you to it. -- Dan ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev
RE: how to preserve saved queries on i2b2 refresh?
The short answer, for HERON at KUMC, is: * Saved queries are supported indefinitely. * We mostly try to evolve terminologies gracefully, * but sometimes we don't bother and we let queries with those terms break. * Saved patient sets are supported for 3 months. To elaborate on the honest broker and version control conversation... i2b2 stores both queries and patient sets. How to preserve them across refreshes? Essentially, back up and restore the relevant tables and schemas (PM, QT, workplace). For (some) details, see HeronLoad#Productioncut-overhttps://informatics.kumc.edu/work/wiki/HeronLoad#Productioncut-over. Do patient ids change? yes; we regenerate them every month, which invalidates any saved patient sets. This was rude to users who would build a patient set on the 30th, only to find it's horked the next day. Our approach to this honest broker issue is: we update patient sets, filling in the corresponding new patient_nums, but only for three months; if we did it indefinitely, then the patient_num could serve as a long-term patient identifier, largely defeating of its privacy protections. (for reference: ticket #699, July 2013 sappa releasehttps://informatics.kumc.edu/work/blog/heron-sappa-update) And yes, there's a lot to talk about w.r.t. ontology version control; we have a dozen or so tickets tagged terminology-evolutionhttps://informatics.kumc.edu/work/query?keywords=%7Eterminology-evolutioncol=idcol=summarycol=keywordscol=statuscol=ownercol=typecol=priorityorder=priority (only about half published :-/). If a user re-runs a query and the terminology has changed so that some of the query's terms are no longer there, it fails (with no clue to the user what's going on, either :-/). So in cases such as #1955https://informatics.kumc.edu/work/ticket/1955 where our notes tree had a clumsy extra ROOT node... Choices include: * Get rid of it * Let any existing queries that use terms below it in the hierarchy break. * Preserve old queries by maintaining a hidden tree (as for old diagnosis terms in #441 and source:heron_load/concepts_merge.sql#L278https://informatics.kumc.edu/work/browser/heron_load/concepts_merge.sql#L278 ) * Keep it In that case, ... we decided not to worry about preserving old queries, whereas when we refined our diagnosis terms, we did keep the old terms, but with c_visualattributes set to hide them. p.s. I didn't ask the question; I think the question was from Angela or Alex in San Antonio. I just brought it here for discussion from the meeting notes. -- Dan From: Wanta Keith M [kwa...@uwhealth.org] Sent: Tuesday, December 02, 2014 6:27 PM To: Dan Connolly; gpc-dev@listserv.kumc.edu Subject: RE: how to preserve saved queries on i2b2 refresh? Very good question Dan! UW Madison currently disables old queries on each i2b2 refresh if our concepts change. So we just do it for each release for now to be safe. We are not handling patient ids (or any ids) correctly yet after a refresh. This is an ice breaker for a great honest broker and ontology version control conversation. From: gpc-dev-boun...@listserv.kumc.edu [mailto:gpc-dev-boun...@listserv.kumc.edu] On Behalf Of Dan Connolly Sent: Tuesday, December 02, 2014 1:48 PM To: gpc-dev@listserv.kumc.edu Subject: how to preserve saved queries on i2b2 refresh? I didn't notice this question in today's meeting notes: ยง Question: on re-deploy to i2b2, how to preserve saved queries? Do patient IDs, etc change. Does i2b2 re-run saved queries or use stored patient sets with old ids? Here's hoping for time to answer... or maybe Nathan will beat me to it. grumble. We have extensive internal documentation on this issue; I wish I could just point you to it. -- Dan ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev
how to preserve saved queries on i2b2 refresh?
I didn't notice this question in today's meeting notes: * Question: on re-deploy to i2b2, how to preserve saved queries? Do patient IDs, etc change. Does i2b2 re-run saved queries or use stored patient sets with old ids? Here's hoping for time to answer... or maybe Nathan will beat me to it. grumble. We have extensive internal documentation on this issue; I wish I could just point you to it. -- Dan ___ Gpc-dev mailing list Gpc-dev@listserv.kumc.edu http://listserv.kumc.edu/mailman/listinfo/gpc-dev
