I spoke too loosley! Maybe not short term. What I was trying to express was a step in an overall direction, e.g., to truly understand a clinical encounter, this seemed to me a natural pre-req. These are my goals with NLP and may not be ctakes goals. I joined ctakes as a step in the understanding requisite to head toward that goal.
Anyways, well see what happens! Im going to see if I can generate a gold standard list I can donate to ctakes, annotated by residents, by which the performance of any approach someone does manage to come up with, myself or otherwise, can be compared. Does such a list exist? Ive heard it mentioned in some of the pubmed articles, but first blush ive seen no publicaclly available set of annotated notes. Maybe each of your institutions has their own propriety lists. Walter Reed, where I work primarily, I havent heard of anything like that. Jg — Sent from Mailbox for iPhone On Mon, Nov 4, 2013 at 8:29 PM, Finan, Sean <sean.fi...@childrens.harvard.edu> wrote: > Hi John, > as the simplest answer to your comment/request: >> I'd be interested in hearing more of what you meant by: " - if not >> completely necessary for any real clinical use of nlp". > I'll answer with your own words: >> a good problem list, whether the physician admits to it or not, is >> interpretation problem-number-one. > There was nothing deep in my writing. > I am a little confused by your statement: >> In the short-term, any NLP wanting to suggest further workup on this man >> would need to a) recognize those features of the HPI and b) prioritize the >> TB workup! > I don't know if anybody has a short-term goal for an nlp tool that makes a > diagnosis (a) or suggests a procedure (b). That seems to be a very long-term > goal for software that goes beyond the processing of natural language in the > note. I may be misreading what you wrote. > I understand your example, and like the ideas of parsing a problem list (if > explicit) or extracting a problem list (if not explicit). These are what I > would say could be immediate goals for nlp. At this time I do not know of > any special problem list section parsing - in fact, cTakes does not handle > formatted lists / tables. Summarization of patient information (extraction > of a problem list and other) from unstructured text is already a big goal. > Reordering a problem list, as well as clumping, would require quite a bit > more than nlp - a database and intelligent decision-making. I'm not saying > that an nlp group would not love to tackle such a matter, just that it spills > outside the domain. > I hope that I am starting to get on the same page, and I am enjoying this > chat - it is different from my normal engagements, which is always nice. > Cheers, > Sean > From: John Green [mailto:john.travis.gr...@gmail.com] > Sent: Monday, November 04, 2013 5:30 PM > To: Finan, Sean > Cc: dev@ctakes.apache.org > Subject: Re: Sundry; Problem Lists > Thank you Sean for taking the time to respond to me, it was much appreciated. > I'm learning a lot about a lot. >>I briefly discussed the first idea (acute vs. historical) with another >>physician (after you brought it up) and there was concurrency that such a >>feature would be extremely useful - if not completely necessary for any real >>clinical use of nlp. I think that if temporal parsing ever becomes finite >>enough with respect to the time of an event relative to the time of the note >>(DocTimeRel) or with proper narrative containers, then this becomes a >>possible use case. I mention this in a weak attempt to pull the nlpers into >>the discussion ... > I'd be interested in hearing more of what you meant by: " - if not completely > necessary for any real clinical use of nlp". I may be showing my lack of > knowledge here again, or I may have miscommunicated in the first instance: a > good problem list, whether the physician admits to it or not, is > interpretation problem-number-one. Take this example of a "History of Present > Ilness" in physician lingo: I come in with a cough, I have a sick child at > home with a cough, I'm also 60 years old and a bad diabetic and a recent lab > value showed an A1C of 9. Further, I'm also a traveler and I just came back > from visiting my cousin in (some country endemic with tuberculosis). Of > course, all of the above may be in a narrative that includes complex story > features, that the physician may or may not have included in the free-text > note. "Mr X is a 60 yo man with a known history of CAD and DMII. Patient > states he came home and had a cough. He further states that his daughter has > a cough. He recently returned from a country in which he had regular contact > with people with TB. He expresses concern and anxiety over this." Well, our > problem list is above (Cough, Sick contact at home (viral), Sick contact > abroad (TB), A1C of 9). In the short-term, any NLP wanting to suggest further > workup on this man would need to a) recognize those features of the HPI and > b) prioritize the TB workup! So the modified by priority problem list would > be 1) Cough 2) TB exposure ... etc. Clumping could ensue. Also, for a > "longitudinal" problem list, one that tracked across clinical encounters, > only the TB exposure and maybe a "history of poorly controlled diabetes" > would need to continue on in the patients history. Certainly a sick child at > home would not (what I meant by acute vs longitudinal problem lists). > Thanks for the conversation Sean, > Sincerely, > John > On Mon, Nov 4, 2013 at 12:15 PM, Finan, Sean > <sean.fi...@childrens.harvard.edu<mailto:sean.fi...@childrens.harvard.edu>> > wrote: > Excellent! By the by, I know next to nothing about nlp - I'm just a software > developer that (for some reason) jumped down this (nlp) particular rabbit > hole. When it comes to nlp background, research, state and direction I'm > hoping that somebody much more knowledgable than I will jump in. >>after a thorough pubmed search, no one seems to have tried to build problem >>lists for ACUTE encounters, only as extensions to a past medical history > I''m really glad that we have a truly novel road on which to travel. >> I seem to be interested in a current encounter (the now) [as opposed to] >> the longitudinal problem list (the ever). > I think that is a great as both a challenge and possible tool, as well as > your thought on >> prioritization, eg enumeration from most important to least, as well as >> clumping > I briefly discussed the first idea (acute vs. historical) with another > physician (after you brought it up) and there was concurrency that such a > feature would be extremely useful - if not completely necessary for any real > clinical use of nlp. I think that if temporal parsing ever becomes finite > enough with respect to the time of an event relative to the time of the note > (DocTimeRel) or with proper narrative containers, then this becomes a > possible use case. I mention this in a weak attempt to pull the nlpers into > the discussion ... >> This is probably well known stuff > Bad assumption ... insert emoticon here ... >>working back from the known natural history of diseases would possibly be a >>route to a solution. > Now that is a challenge! > Cheers for the inspiration and enthusiasm, > Sean > ________________________________ > From: John Green > [john.travis.gr...@gmail.com<mailto:john.travis.gr...@gmail.com>] > Sent: Monday, November 04, 2013 10:45 AM > To: Finan, Sean > Subject: RE: Sundry; Problem Lists > Oh goodness no, I didnt think that at all! Im so new to the field of NLP, > anything and everything helps and is appreciated. Heck, im just now learning > to understand Markov chains. > An additional thought: after a thorough pubmed search, no one seems to have > tried to build problem lists for ACUTE encouters, only as extensions to a > past medical history. I think this would be a very fruitful avenue. It could > easily be scored against a gold standard medical resident list for a few > hundred patients across depth and acuity. > Just thinkin out loud, bouncing ideas off those who know more than I! > Jg > - > Sent from Mailbox<https://www.dropbox.com/mailbox> for iPhone > On Mon, Nov 4, 2013 at 9:24 AM, Finan, Sean > <sean.fi...@childrens.harvard.edu<mailto:sean.fi...@childrens.harvard.edu>> > wrote: > Hi John, > I hope that you didn't think that I was belittling your ideas or saying that > anything has been done (and done). I was just throwing in two resources for > further thought. You have brought forward some great applications for cTakes > and nlp! > Sean > ________________________________________ > From: John Green > [john.travis.gr...@gmail.com<mailto:john.travis.gr...@gmail.com>] > Sent: Thursday, October 31, 2013 7:26 PM > To: dev@ctakes.apache.org<mailto:dev@ctakes.apache.org> > Subject: RE: Sundry; Problem Lists > Last point: I seem to be interested in a current encounter (the now) and > diagnosis, the article seems to be interested in an arguably just as useful > tool, the longitudinal problem list (the ever), though very different I would > think in approach. > Thoughts? > Jg > - > Sent from Mailbox for iPhone > On Thu, Oct 31, 2013 at 7:22 PM, John Green > <john.travis.gr...@gmail.com<mailto:john.travis.gr...@gmail.com>> > wrote: >> Sean - quick note: after looking at the above two resources, a couple of >> points. The first resource confirms what I expected, that the vocabulary >> exists in ctakes. The second confirms what I suspected: that novel >> approaches to ordering and identification of top members of a problem list >> are needed. Namely, that the vocabulary may be there, but thats only a tenth >> of the battle. Your second great resource you sent me acknowledges this - >> that prioritization, eg enumeration from most important to least, as well as >> clumping, are the true battle. >> A point of clarification on my end: it would be interesting to see what >> could be added on top of existing ctakes in order to facilate a solution to >> the second problem - clumping and prioritizing. (For instance, from the >> second article, an acute process may have nothing todo with the past medical >> history and if an algorithm were concerned with all members as equals, it >> would miss the issue at hand). >> Just as a thought: working back from the known natural history of diseases >> would possibly be a route to a solution. >> This is probably well known stuff, so please forgive my ignorance if its all >> been done/thought of before. >> Again, the two links were very helpful, thank you. >> Jg >> - >> Sent from Mailbox for iPhone >> On Thu, Oct 31, 2013 at 2:04 PM, Finan, Sean >> <sean.fi...@childrens.harvard.edu<mailto:sean.fi...@childrens.harvard.edu>> >> wrote: >>> I don't know if what I write below truly applies to the discussion, but >>> here it is. >>>>much of a problem list definition may already be contained to varying >>>>degrees >>>> in existing cTakes databases. >>> The UMLS does provide a problem list, but I haven't looked at it. >>> http://www.nlm.nih.gov/research/umls/Snomed/core_subset.html >>> This might be a paper of interest to you: >>> http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2655994/ >>> It discusses the use of nlp to create something like a problem list. >>> Sean >>> ________________________________________ >>> From: John Green >>> [john.travis.gr...@gmail.com<mailto:john.travis.gr...@gmail.com>] >>> Sent: Thursday, October 31, 2013 12:02 PM >>> To: dev@ctakes.apache.org<mailto:dev@ctakes.apache.org> >>> Subject: Re: Sundry >>> Pei and Tim - Good questions. >>> The bottom line is that OPQRST is the algorithm that every clinician uses >>> to characterize the history of a sign, symptom or constellation of >>> symptoms. Each letter has multiple meanings, but generally they're grouped. >>> O for onset, was it quick or slow in onset, P for palliative or provoking >>> phenomenon, that is, does tylenol make it better? Does it feel better when >>> you lean forward? Is it worse with standing? Q is the quality, generally, >>> though I could give more examples of each Ill keep it brief from here, R is >>> generally region or radiation of the pain and or sign, S is the severity, >>> and T is the time course, is it intermittent? When it happens, how long >>> does it last for? I could send documents used to teach new clinicians to >>> better comprehend for anyone interested. >>> OPQRST, while most residents would assume it is only for teaching new >>> clinicians, as Tim said, is a useful tool at all levels. Great clinicians, >>> and I work with some great senior folks, use this everyday. The idea that >>> it is only for teaching is founded on two things: one, that it doubles as a >>> structured mnemonic for characterizing signs and symptoms and two, that >>> everyone so far ingrains this into their clinical skill set, unless they >>> are geared toward teaching, they, after the basic level, never think about >>> it again! Caveat: many good clinicians will tell you to keep it algorithmic >>> so that you're systematic and do not overlook details. >>> What is it's application to ML? Obviously the furthest desired end-state >>> for NLP like cTakes would be understanding a clinical encounter to such a >>> nuanced level that detailed diagnoses could be considered along with >>> treatment plans. While I only know what I've read in Artificial >>> Intelligence: A Modern Approach and picked up from friends over the years >>> who were good knowledgeable in this field, I feel that OPQRST would be a >>> huge benefit toward beginning to outline the problem of more rigorous ML >>> characterization of the clinical narrative. >>> The utility of OPQRST may not still be entirely clear to those who have >>> never been presented with a clinical encounter. Let me try one more stab: >>> Take the classic example of chest pain. A man comes to the ER with chest >>> pain. Is the onset quick? Yes doc, it was all of a sudden. This might >>> support a diagnosis of, say, MI, aortic dissection, pulmonary embolism, but >>> less likely someone would call GERD sudden. Palliative or provoking >>> features? Well, when I take 8 antacids it gets better (GERD), or, When I >>> take my wifes nitroglycerine it got better for a little while (angina), or, >>> when I took my wifes nitroglycerine it did nothing (pericarditis?). >>> Quality: Is it stabbing? Ya doc, its stabbing (less likely MI). Is it >>> crushing? Like an elephant on your chest? Ya doc, that's it. (more likely >>> MI), and so on. >>> Now of course, cTakes could be used for a real life encounter like this >>> (middleware) at some point, but likely it would be taking a history and >>> proposing a diagnosis (middleware again Tim, yes). But the point is, the >>> first steps toward knowing what were dealing with at the historical level >>> is centered around OPQRST, and it just occurred to me to ask what we >>> thought about the feasibility of something like that. >>> In retrospect, it may be too tough, but at some point it would need done, >>> just as much as a clinician must learn it. >>> One final point: problem lists. These are absolutely essential to any >>> clinician in making a diagnosis. Again, often times, they dont think about >>> it, but they use them. When writing the above it occurred to me: much of a >>> problem list definition may already be contained to varying degrees in >>> existing cTakes databases. It would be an interesting and worthwhile paper, >>> I think, to see how well cTakes compiled problem lists matched Medical >>> Students, Residents, and Attending physician's problem lists. If anyone is >>> interested in this line of thought, I would be interested in collaborating. >>> It would be very easy, and the data may actually already exist to compare. >>> Forgive me if its already been done, but, if it hasnt, then it would go a >>> long way toward proving cTakes efficacy in regards to high-order processes. >>> And if it hasnt been done and someone does it at a later date, please, send >>> me an email to the paper! >>> JG >>> On Wed, Oct 30, 2013 at 10:08 AM, Tim Miller < >>> timothy.mil...@childrens.harvard.edu<mailto:timothy.mil...@childrens.harvard.edu>> >>> wrote: >>>> Thanks for bumping this Pei, it reminds me I meant to respond to it. >>>> >>>> The OPQRST does sound like a great ML project. At a glance I might think a >>>> sequence model over sentences (like a CRF) would be a good model. >>>> But I'm wondering what the end use case is? Is it for teaching OPQRST to >>>> new clinicians? Or maybe as a sort of middleware for other projects where >>>> it might be a useful feature? Without a physician's intuition I sometimes >>>> suffer from a failure of imagination on these things. >>>> >>>> Tim >>>> >>>> >>>> >>>> On 10/30/2013 09:59 AM, Chen, Pei wrote: >>>> >>>>> Hi John, >>>>> I was away for a little bit and finally got a chance to catch up on >>>>> emails... >>>>> >>>>> 2) I work for the DoD and have latched on to several IRB approved >>>>>> projects >>>>>> within that community where Ill be using cTakes, though minimally at >>>>>> first. >>>>>> This is just a statement, a bug in the ear of the community of what >>>>>> people >>>>>> are up to. >>>>>> >>>>> This is really news! Looking forward to hearing more... >>>>> >>>>> has anyone considered (and maybe the components already do this in some >>>>>> way I >>>>>> haven't explored yet - time is ever limited) adding an OPQRST classifier? >>>>>> >>>>> I'm not too familiar on how OPQRST would be determined from the patient's >>>>> record. >>>>> Just curious, how is it currently determined manually now? Is it a >>>>> single score determined by a formula/rule(s)? >>>>> Seems like another good use case for cTAKES output-- clinically focused. >>>>> --Pei >>>>> >>>> >>>>
