uncertainty in medical problem solving and decision making (Was: Dr R LONJON Confidence indicator !
On Apr 27, 2005, at 8:19 AM, Arild Faxvaag wrote: We could say that physicians _infer_ diagnostic hypotheses based on - knowledge of the tentative underlying disease, - the patients subjective experiences - phenomena registered in the patients body In any case it is a subjective statement and is a professional opinion based on more (lab results, x-rays) or less (patient history) objective data. Phrases such as cannot be exluded might be due to, probably, definitely, beyond doubt are statements of probability of the inferrence being correct (and what to do next). Inferrences expressed in the subjective statements documenting the treatment of the patient. Can one say that diagnoses belong to the class of statements whereas the disease itself belong to the class of natural phenomena? Disease is an abstraction of reality that for the moment, for the next decision is considered to represent the reality about the health of the patient. Diagnosis is the professional but subjective opinion about a disease of a patient. There is a continuum: Real pathological, fysiologiscal phenomena in a patient. Certain manifestations of these phenomena. That are (or are not) experienecd by the patient of an other person. The arrousal of distress, anxiety, etc, triggering a visit to a physisian. What is said (or not) about the manifestations of the phenomena during the visit. And how it is said. How it is measured and documented. What is understood of what was said or measured about the manifestations of the phenomena. How all this was mached to the state-of-the art knowledge, or interpreted in the context of a limited amount of available knowledge. What was recorded about all these steps above. How the same person (or others) interpret the recorded 'facts' at a later stage So what do we record in an EHR? And what do we interpret readingan EHR? Then ... What is certain? And what is uncertain? Certain or uncertain in what domain, in what line above, at what level of the whole described continuum? In ?25% of the extremely wel researched patients in one University Hospital we not diagnosed correctly during their life time. As could be concluded after an autopsy. So what do we really know about disease and complaints? What is certainty? What is it refering to? Do we understand this mine field well enough? The diagnosis establishes a relation between the subjective experiences / phenomena and the disease that induces those symptoms and findings. Example: Experiences and phenomena: Pain in the wrist joints, feeling of joint stiffness, joint tenderness, joint swelling, elevated sedimentation rate. Diagnostic inferrence: Rheumatoid arthritis. Relation: Might be induced by/due to Can statements of probability be considered statements regarding the strength of these relations?? This is what they are at best. Comments on this? -- next part -- An HTML attachment was scrubbed... URL: http://lists.openehr.org/mailman/private/openehr-technical_lists.openehr.org/attachments/20050528/26059551/attachment.html
The Uncertainty Decision was: Dr R LONJON Confidence indicator !
On May 7, 2005, at 3:12 PM, Thomas Beale wrote: ...so it seems to me that the indicator of what to do next when a differential diagnosis is recorded relates strongly to the innate characteristics of the conditions recorded, not just the doctor's opinion of how likely it might be. If angina pectoris is a possible diagnosis for burning chest pain at 5%, with the most probable diagnosis (in the opinion of the physician) being gastric reflux at 95%, and it is a 55-yo with a family history of coronary heart disease, I presume that the angina pectoris possibility is the one that drives the next steps? How are the confidences really decided? In all cases what is recorded is the personal opinion of the healthcare provider. -- next part -- An HTML attachment was scrubbed... URL: http://lists.openehr.org/mailman/private/openehr-technical_lists.openehr.org/attachments/20050527/ddf67ffa/attachment.html
The Uncertainty Decision was: Dr R LONJON Confidence indicator !
P? 7. mai. 2005 kl. 15.12 skrev Thomas Beale: Gerard Freriks wrote: The EHR is not invented to describe the real actual health status of the patient. It is there to document what clinicians deemed important to say ABOUT the health status of the patient. It always is an opinion of a professional about something. yes, hopefully we all agree with this philosophy. I can agree that the doctors hypothesis is an opinion, but those parts of the EHR that are pure descriptions of phenomena and symptoms, plans and descriptions of actions are not. But we need to add (contradict me if I'm wrong;-) that it is what clinicians wanted to say which they deemed relevant to next steps - either diagnostic or intervention. What to do next is not just based on the doctor's confidence about what the symptoms might mean, but also on: - the urgency of treatment of that condition (cases like cerebral meningitis, malaria...) - the severity of the condition (e.g. cystic fibrosis) - the severity of the consequences of the condition on others (CF, huntington's, ...) The issues here are - the severity of the disease - the course of the disease if not treated - the potential benefits of the intervention - the probability of the patient actually achieving these benefits - the cost and complexity of the intervention - the potential side effects of the intervention - the probability of these to develop - the patient's preferences (some patients are risk takers, others are not) one can also add - the need to convince the patient (replace the patient's hypothesis with that of the physician). - the need to maintain the patients trust in the provider (unless the patient might withdraw the care mandate / not give his consence to the plans suggested by the physician). ...so it seems to me that the indicator of what to do next when a differential diagnosis is recorded relates strongly to the innate characteristics of the conditions recorded, not just the doctor's opinion of how likely it might be. If angina pectoris is a possible diagnosis for burning chest pain at 5%, with the most probable diagnosis (in the opinion of the physician) being gastric reflux at 95%, and it is a 55-yo with a family history of coronary heart disease, I presume that the angina pectoris possibility is the one that drives the next steps? How are the confidences really decided? How are we to bridge the gap between the physician-recorded confidence factor and the total list of factors which drive the next steps? What do we need in the EHR? Is this just a decision support problem (where the physician will be performing the decision support)? Very briefly,, here are some factors (using terminology adapted from risk analysis): - Being healthy is something of high value. - Posessing a (unexplained) health problem implies being exposed to a potential threat to ones health. - When the patients seek a doctor he gives responsibility to the doctor and thereby partially transfer the risk to him (partially the health care person, partially the organisation who employs this person). - Diagnostics can be considered risk exploitation. - Therapeutics can be considered risk managment. - The patients confidence in the provider is a prerequisite for both diagnostics and therapeutics and is therefore (from the perspective of the provider) something which has high value in its own (and therefore must be maintained). - Services from health personell to patients are therefore justified for two reasons -- to maintain the patients health -- to maintain the patients trust in the provider This leads to the perspective on the EHR system as a tool for exploitation and managment of risks that can harm the patient or his trust in the provider. Can this shed new light on the issue of the confidence indicator? Arild Faxvaag -- next part -- A non-text attachment was scrubbed... Name: not available Type: text/enriched Size: 3978 bytes Desc: not available URL: http://lists.openehr.org/mailman/private/openehr-technical_lists.openehr.org/attachments/20050509/c362aeec/attachment.bin
The Uncertainty Decision was: Dr R LONJON Confidence indicator !
On Sat, May 07, 2005 at 02:12:45PM +0100, Thomas Beale wrote: If angina pectoris is a possible diagnosis for burning chest pain at 5%, with the most probable diagnosis (in the opinion of the physician) being gastric reflux at 95%, and it is a 55-yo with a family history of coronary heart disease, I presume that the angina pectoris possibility is the one that drives the next steps? How are the confidences really decided? If it's a 55-yo with a family history of coronary heart disease and the doctor thinks angina pectoris is at 5% while gastric reflux is at 95% then it is either a failure of the doctor to get his probabilites straight - or else the doctor is truly clueful (eg knows the patient very well) - in which case, yes, the gastric reflux would be driving the next steps. How are we to bridge the gap between the physician-recorded confidence factor and the total list of factors which drive the next steps? In such cases I usually record IMO this but that not r/o yet hence act on this but also do foo to differentiate. In clear text. Karsten -- GPG key ID E4071346 @ wwwkeys.pgp.net E167 67FD A291 2BEA 73BD 4537 78B9 A9F9 E407 1346 - If you have any questions about using this list, please send a message to d.lloyd at openehr.org
The Uncertainty Decision was: Dr R LONJON Confidence indicator !
Hi, I agree with Thomas, probably because we are engineers and ask ourselves If they don't record this information for further action, why do they record it anyway ?. I can perfectly understand the way Gerard thinks to it, in an EHRcom way : I use this EHR for myself, and I can send you a part of MY EHR record to complete yours (sorry Gerard if it seems over-simple). From my own point of view (at least for the kind of systems I am working on), the members of a patient's health team are contributors on a common working place, and, (if we don't ask them to be God) we expect for more involvment and accuracy in the process. Cheers, Philippe Thomas Beale wrote: Gerard Freriks wrote: The EHR is not invented to describe the real actual health status of the patient. It is there to document what clinicians deemed important to say ABOUT the health status of the patient. It always is an opinion of a professional about something. yes, hopefully we all agree with this philosophy. But we need to add (contradict me if I'm wrong;-) that it is what clinicians wanted to say which they deemed relevant to next steps - either diagnostic or intervention. What to do next is not just based on the doctor's confidence about what the symptoms might mean, but also on: - the urgency of treatment of that condition (cases like cerebral meningitis, malaria...) - the severity of the condition (e.g. cystic fibrosis) - the severity of the consequences of the condition on others (CF, huntington's, ...) ...so it seems to me that the indicator of what to do next when a differential diagnosis is recorded relates strongly to the innate characteristics of the conditions recorded, not just the doctor's opinion of how likely it might be. If angina pectoris is a possible diagnosis for burning chest pain at 5%, with the most probable diagnosis (in the opinion of the physician) being gastric reflux at 95%, and it is a 55-yo with a family history of coronary heart disease, I presume that the angina pectoris possibility is the one that drives the next steps? How are the confidences really decided? How are we to bridge the gap between the physician-recorded confidence factor and the total list of factors which drive the next steps? What do we need in the EHR? Is this just a decision support problem (where the physician will be performing the decision support)? He, himself, always makes statements with varying degrees of certainty. Physicians are no gods that know everything. What? And I thoughtoh no, my whole world is shattered...:-) - thomas - If you have any questions about using this list, please send a message to d.lloyd at openehr.org - If you have any questions about using this list, please send a message to d.lloyd at openehr.org
The Uncertainty Decision was: Dr R LONJON Confidence indicator !
Gerard Freriks wrote: The EHR is not invented to describe the real actual health status of the patient. It is there to document what clinicians deemed important to say ABOUT the health status of the patient. It always is an opinion of a professional about something. yes, hopefully we all agree with this philosophy. But we need to add (contradict me if I'm wrong;-) that it is what clinicians wanted to say which they deemed relevant to next steps - either diagnostic or intervention. What to do next is not just based on the doctor's confidence about what the symptoms might mean, but also on: - the urgency of treatment of that condition (cases like cerebral meningitis, malaria...) - the severity of the condition (e.g. cystic fibrosis) - the severity of the consequences of the condition on others (CF, huntington's, ...) ...so it seems to me that the indicator of what to do next when a differential diagnosis is recorded relates strongly to the innate characteristics of the conditions recorded, not just the doctor's opinion of how likely it might be. If angina pectoris is a possible diagnosis for burning chest pain at 5%, with the most probable diagnosis (in the opinion of the physician) being gastric reflux at 95%, and it is a 55-yo with a family history of coronary heart disease, I presume that the angina pectoris possibility is the one that drives the next steps? How are the confidences really decided? How are we to bridge the gap between the physician-recorded confidence factor and the total list of factors which drive the next steps? What do we need in the EHR? Is this just a decision support problem (where the physician will be performing the decision support)? He, himself, always makes statements with varying degrees of certainty. Physicians are no gods that know everything. What? And I thoughtoh no, my whole world is shattered...:-) - thomas - If you have any questions about using this list, please send a message to d.lloyd at openehr.org
The Uncertainty Decision was: Dr R LONJON Confidence indicator !
Arild and Tim This is clearly an issue. In the CIP project the group wanted to be able to say that a diagnosis was a working diagnosis. We have archetyped a number of concepts that I think will enable the clinician to express these levels of uncertainty without resorting to confidence ratings on all entries in the record. Arild has shown that you could not possibly do a mastectomy without rating your certainty at 100% - or you will be sued. And not treating a pneumonia in a newborn with a certainty of only 20% will probably get you in trouble. These sort of explicit ratings are - in my opinion - very problematic. The solution lies in the recording constructs used for many years: 1. To express differential diagnoses (with or without probabilities) and to note key excluded diagnoses as well. 2. To express a diagnosis as a problem (such as lump in left breast) even if the likelihood of cancer is 100% clinically until the histology is returned. 3. To be able to label a diagnosis as a working diagnosis - ie it is likely enough to warrant the current management - but not certain. Appendicitis is a good example. So the archetypes for problem, problem-diagnosis (specialised) and differential diagnosis should meet these needs. Comments? Sam Tim Cook wrote: While it might be an interesting exercise for us to record how confident a clinician was at the time of recording a diagnosis, it will have no impact on the health care of that patient. If we were to do this would we ask them to do so in sarcasm10% steps, 5% steps or .01% steps/sarcasm? I assert that any one of these would seriously impact the usability of an EHR in a negative manner and would result in the clinician taking the option that presents the least liability on their part. So back to the short answer above.is it really relevant to assert ANY confidence factor in the EHR? My opinion is that there indeed is highly relevant to assert a confidence factor in the EHR. ln decision analysis one talks about treatment thresholds for diagnostic uncertainity as the probability of disease at which the expected value of treatment and no treatment are exactly equal, and ne ither option is clearly preferable. (Hunik and Glasziiou Decision making in health and biomedicine). Factors influencing the treatment threshold are the expected benefit and the expected harm of the treatment. Example: Treatment threshold is much lower for pneumonia (treatment: penicillin) than for cancer of the left mamma (treatment: Mastectomy) Thus: How confident a clinician is at the time of recording a diagnosis has high impact on the health care of that patient. Comments on this? regards, Arild Faxvaag - If you have any questions about using this list, please send a message to d.lloyd at openehr.org
The Uncertainty Decision was: Dr R LONJON Confidence indicator !
Hello, I read opinions expressed on the topic. This question is important in France. The government took the decision that all citizen is going to have an electronic medical file.personal (DMP acronym) In principle all physicians with the authorization of the patient will have an access to this medical file for me it is about a medical file published a little like a weblog (to private and controlled acc?s) It is completely different of the electronic medical file that every physician must create and hold up to date for his/her/its patient in his/her/its cabinet. we call it the software profession.( logiciel m?tier in french ) This DMP should receive information exported from the software profession of the physician. The difficulty is to decide: 1 - what information must be published, 2 - this information is it reliable, so that another physician can use him and not to ask for a new exam 3 - if the physician producer of information, has a space of liberty, so that his/her/its responsibility implication is not systematically.? The solution would be can be to differentiate well: 1 - an information validated by the physician and that gives him the opposable information statute. He/it accepts to hire his/her/its responsibility. It is an information that is certified by documents as the imagery, the biopsy, the biologic analyses. 2 - an information proposed by the physician and that gives him the likely, possible information statute, but of which the level of certainty is not sufficient to have the opposable information statute. In this case the responsibility of the physician, be able to not be put in reason, while using this information no validated like proof. It is a legislative and legal probl?me, that is different of a computer analysis, but that is real. Indulgence for my English and thank you. Dr R LONJON France Selon Gerard Freriks gfrer at luna.nl: Sam, I agree. Suggestion In otherwords any clinical (or non-clinical) concept model must be able to express the view of the author about certainty. 3 states are sufficient for starters: likely (as default) not-likely certain When a person attaches new information to the EHR and is of the opinion that whole or parts of a received extract (or EHR) need an other qualifyer then via versioning he must be able to annotate this by adding his beliefs about certainty. Gerard -- private -- Gerard Freriks, arts Huigsloterdijk 378 2158 LR Buitenkaag The Netherlands +31 252 544896 +31 654 792800 On 27 Apr 2005, at 23:25, Sam Heard wrote: Arild and Tim This is clearly an issue. In the CIP project the group wanted to be able to say that a diagnosis was a working diagnosis. We have archetyped a number of concepts that I think will enable the clinician to express these levels of uncertainty without resorting to confidence ratings on all entries in the record. Arild has shown that you could not possibly do a mastectomy without rating your certainty at 100% - or you will be sued. And not treating a pneumonia in a newborn with a certainty of only 20% will probably get you in trouble. These sort of explicit ratings are - in my opinion - very problematic. The solution lies in the recording constructs used for many years: 1. To express differential diagnoses (with or without probabilities) and to note key excluded diagnoses as well. 2. To express a diagnosis as a problem (such as lump in left breast) even if the likelihood of cancer is 100% clinically until the histology is returned. 3. To be able to label a diagnosis as a working diagnosis - ie it is likely enough to warrant the current management - but not certain. Appendicitis is a good example. So the archetypes for problem, problem-diagnosis (specialised) and differential diagnosis should meet these needs. Comments? Sam -- - If you have any questions about using this list, please send a message to d.lloyd at openehr.org
The Uncertainty Decision was: Dr R LONJON Confidence indicator !
Tim Cook wrote: While it might be an interesting exercise for us to record how confident a clinician was at the time of recording a diagnosis, it will have no impact on the health care of that patient. If we were to do this would we ask them to do so in sarcasm10% steps, 5% steps or .01% steps/sarcasm? I assert that any one of these would seriously impact the usability of an EHR in a negative manner and would result in the clinician taking the option that presents the least liability on their part. So back to the short answer above.is it really relevant to assert ANY confidence factor in the EHR? My opinion is that there indeed is highly relevant to assert a confidence factor in the EHR. ln decision analysis one talks about treatment thresholds for diagnostic uncertainity as the probability of disease at which the expected value of treatment and no treatment are exactly equal, and ne ither option is clearly preferable. (Hunik and Glasziiou Decision making in health and biomedicine). Factors influencing the treatment threshold are the expected benefit and the expected harm of the treatment. Example: Treatment threshold is much lower for pneumonia (treatment: penicillin) than for cancer of the left mamma (treatment: Mastectomy) Thus: How confident a clinician is at the time of recording a diagnosis has high impact on the health care of that patient. Comments on this? regards, Arild Faxvaag -- next part -- A non-text attachment was scrubbed... Name: not available Type: text/enriched Size: 1527 bytes Desc: not available URL: http://lists.openehr.org/mailman/private/openehr-technical_lists.openehr.org/attachments/20050427/be6e62a1/attachment.bin
uncertainty in medical problem solving and decision making (Was: Dr R LONJON Confidence indicator !
Hi Arild, Another site is the MIT Group on Clinical Decision Making: [ http://medg.lcs.mit.edu/ ]. ... a research group dedicated to exploring and furthering the application of technology and artificial intelligence to clinical situations. Because of the vital and crucial nature of medical practice, and the need for accurate and timely information to support clinical decisions, the group is also focused on the gathering, availability, security and use of medical information throughout the human life cycle and beyond ... Unfortunately Patient decision-making receives less emphasis and studies seem to miss some fundamental factors (e.g., it is private) [ http://www.ahrq.gov/research/rtisumm.htm ] Regards! -Thomas Clark Arild Faxvaag wrote: Hi all. This is an important topic. Here are some references / pointers for those who wish to read more: Decision making in health and medicine. Integrating evidence and values Myriam Hunink and Paul Glasziou Cambridge university press (ISBN 0 521 77029 7) Society for Medical Decision Making: http://www.smdm.org/ I also recommend journal articles written by Wimla L Patel (Colombia university, New York), for instance: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrievedb=pubmeddopt=Abstractlist_uids=11418539 (A primer on aspects of cognition for medical informatics) regards arild Faxvaag P 22. apr. 2005 kl. 07.42 skrev Gerard Freriks: -1- Almost never a diagnosis is 100% certain. -2- Almost always a test result has uncertainty attached to it -3- Many times a conclusion is reached based on many uncertain and conflicting facts -4- Quite often a condition, a diagnosis, is assumed that gives rise to a treatment. Not indicating that the patient is suffering from this condition but using treatment as a test procedure. Doing nothing is such a test procedure. Eric Wulff (from Danmark) published philisophical texts about health care and these topics. gerard -- private -- Gerard Freriks, arts Huigsloterdijk 378 2158 LR Buitenkaag The Netherlands +31 252 544896 +31 654 792800 On 20 Apr 2005, at 13:58, Thomas Beale wrote: I'm wondering if there is a meta-algorithm of some sort lurking behind the scenes, which takes account of uncertainty in a note, and also severity of non-discounted possibilities, as a way of deciding what to do next. There is undoubtedly published work on this... thoughts? - thomas beale -- Arild Faxvaag associate professor / rheumatologist Adress / Office St.Olavs hospital: Department of Rheumatology, St.Olavs hospital N-7006 Trondheim, Norway Phone Dept of Rheumatology 47 7386 7263 Adress / Office NTNU Norwegian center for electronic patient records research (NSEP) Medisinsk teknisk forskningssenter N-7489 Trondheim Cellphone: 47 9821 6825 http://www.ntnu.no/~arildfa/ (home page NTNU) http://www.usemed.com (weblog on e-medicine) http://www.ehr.ntnu.no/e (Norwegian Centre for Electronic Health Records Research) - If you have any questions about using this list, please send a message to d.lloyd at openehr.org
Dr R LONJON Confidence indicator !
-1- Almost never a diagnosis is 100% certain. -2- Almost always a test result has uncertainty attached to it -3- Many times a conclusion is reached based on many uncertain and conflicting facts -4- Quite often a condition, a diagnosis, is assumed that gives rise to a treatment. Not indicating that the patient is suffering from this condition but using treatment as a test procedure. Doing nothing is such a test procedure. Eric Wulff (from Danmark) published philisophical texts about health care and these topics. gerard -- private -- Gerard Freriks, arts Huigsloterdijk 378 2158 LR Buitenkaag The Netherlands +31 252 544896 +31 654 792800 On 20 Apr 2005, at 13:58, Thomas Beale wrote: I'm wondering if there is a meta-algorithm of some sort lurking behind the scenes, which takes account of uncertainty in a note, and also severity of non-discounted possibilities, as a way of deciding what to do next. There is undoubtedly published work on this... thoughts? - thomas beale -- next part -- A non-text attachment was scrubbed... Name: not available Type: text/enriched Size: 1072 bytes Desc: not available URL: http://lists.openehr.org/mailman/private/openehr-technical_lists.openehr.org/attachments/20050422/bbb58527/attachment.bin
Dr R LONJON Confidence indicator !
Elkin, Peter L., M.D. wrote: Dear Roger and Thomas, We have looked extensively at Multivalued logic for quantitating uncertainty. It turns out that most folks in that world have taking 0 false and one true with a number of discrete, usually equally spaced values in between for uncertainty. After a longwinded go around with a Prof of Philosophical Logic at Princeton (Dr. Graham) We determined that there at least three reproducible types of uncertainty (with good inter-rater reliability) and ~ seven semantic categories. The types are Probable (our guess is around 85% true +/- 5%) and Unlikely (our guess is around 15% true +/- 5%) or Just as likely as not (again our guess is around 50% +/- 15%). These number come from the average PPV of the evidence when a physician Makes a diagnosis and NPV when a physician rules one out. [with appropriate excuses in advance for my engineer's view of clinical things;-] I presume that these values (which seem entirely reasonable to me) were obtained by a statistical study of clinicians' notes? Or interviews? But the problem we are always concerned with is: what does one clinician mean when s/he says probable rheumatoid arthritis? We can't assume it can be translted into 85% +/- 5% can we? The particular physician who said it might habitually and unconsciously put probable all over the place, when they should really put possible. Sam's point of view so far has been: make them enter a number (prompt = % probability of being true or similar). I know that doesn't address the perfectly reasonable need to allow clinical people to write probable, possible etc, so maybe it's not a long term answer. But let's just consider what doing clinical medicine is about: it's just scientific problem-solving. The goal is to fix a problem (with the patient); the method is to iteratively gather information until a conclusion (diag = Rh Arthritis) can be drawn or a decision can be made (commence ibuprofen). Fixing a problem may involve many repetitions of this until the problem is fixed. Now, whenever (lack of ) confidence or uncertainty occurs, it means that we don't have enough information to make a decision or draw a conclusion, at least not the next one in the chain. But we do have an indication of what to do next - usually gather more information. So perhaps the way we view words like possible, likely, probable should be as motivators to perform more actions to reduce the uncertainty. If a doctor writes possible malaria re: a patient just back from a holiday vietnam, with heavy flu-like symptoms, the obvious implication is to do the appropriate microscopy other diagnostic procedures for malaria, to rule it out or otherwise. For most diseases, a diagnostic algorithm or guideline is available, and the physician having used a word implying uncertainty just means that the diagnostic process is currently at some interior node of such a guideline tree. The key question is probably _which_ of the possible next steps to rule out /rule in one of the differential diagnoses to do in which order - i.e. which is cheapest, fastest, most relevant to patient health etc. So my question to clinicians is this: doesn't a note containing possible X, likely Y really imply a differential diagnosis, even if only one of the possibilities is actually noted? If so, it may not matter what the level of uncertainty is so much; what matters (among other things) is the severity of the consequences of any of the possible branches of the differential diagnosis. E.g. if one of the implied or noted branches of a differential diagnosis for a patient presenting fever is malaria, presumably both patient and doctor want to discount it as fast as possible, and pursue the appropriate steps to do so. But if none of the branches is life-threatening, reasonable action may be wait 12 hours and re-assess. The very common situation of infant presenting with fever must present such a quandary daily. I'm wondering if there is a meta-algorithm of some sort lurking behind the scenes, which takes account of uncertainty in a note, and also severity of non-discounted possibilities, as a way of deciding what to do next. There is undoubtedly published work on this... thoughts? - thomas beale - If you have any questions about using this list, please send a message to d.lloyd at openehr.org
Dr R LONJON Confidence indicator !
Tim Churches wrote: Thomas Beale wrote: I'm wondering if there is a meta-algorithm of some sort lurking behind the scenes, which takes account of uncertainty in a note, and also severity of non-discounted possibilities, as a way of deciding what to do next. There is undoubtedly published work on this... This is a very brief but reasonable introduction to Bayesian probability (which includes calculation of utility), which is what I think you are grasping at: http://en.wikipedia.org/wiki/Bayesian_probability Hi Tim, and there are quite a few decision support products based on Bayesian logic as well. But I wonder if they have been applied to the problem of determining next best steps based not just on clinical data so far, but also cost, duration, and perceived severity of consequences of not doing something. And I think that Bayesian products should take as inputs only weightings proven by population studies, whereas physician belief is often supported by informal but often qutie accurate personal experience (i.e. experience of the patient population of the practice). In any case, can we argue that there is no point caring about any finer gradations of true/false than true/false/maybe, as Peter Elkin has said they are doing at Mayo? - thomas - If you have any questions about using this list, please send a message to d.lloyd at openehr.org
Dr R LONJON Confidence indicator !
Dear Thomas, I think we need clinicians to be more precise in these declarations. If we begin to train clinicians that Probable should mean ~85% probability +/- 5% then we will move closer to stability. Although the goal of reducing uncertainty is in general laudible there are some problems that crop up first clinicians are usually only about 90% sure by evidence when they make a diagnosis if looked at from an EBM perspective. Also the path to reduction of uncertainty takes into account what prior data is available, the risk benefit ratio of obtaining each piece of data, and patient preference. Interesting but not easy. Peter Peter L. Elkin, MD Professor of Medicine Mayo Clinic College of Medicine -Original Message- From: owner-openehr-technical at openehr.org [SMTP:owner-openehr-technical at openehr.org] On Behalf Of Thomas Beale Sent: Wednesday, April 20, 2005 6:59 AM To: openehr-technical at openehr.org Subject: Re: Dr R LONJON Confidence indicator ! Elkin, Peter L., M.D. wrote: Dear Roger and Thomas, We have looked extensively at Multivalued logic for quantitating uncertainty. It turns out that most folks in that world have taking 0 false and one true with a number of discrete, usually equally spaced values in between for uncertainty. After a longwinded go around with a Prof of Philosophical Logic at Princeton (Dr. Graham) We determined that there at least three reproducible types of uncertainty (with good inter-rater reliability) and ~ seven semantic categories. The types are Probable (our guess is around 85% true +/- 5%) and Unlikely (our guess is around 15% true +/- 5%) or Just as likely as not (again our guess is around 50% +/- 15%). These number come from the average PPV of the evidence when a physician Makes a diagnosis and NPV when a physician rules one out. [with appropriate excuses in advance for my engineer's view of clinical things;-] I presume that these values (which seem entirely reasonable to me) were obtained by a statistical study of clinicians' notes? Or interviews? But the problem we are always concerned with is: what does one clinician mean when s/he says probable rheumatoid arthritis? We can't assume it can be translted into 85% +/- 5% can we? The particular physician who said it might habitually and unconsciously put probable all over the place, when they should really put possible. Sam's point of view so far has been: make them enter a number (prompt = % probability of being true or similar). I know that doesn't address the perfectly reasonable need to allow clinical people to write probable, possible etc, so maybe it's not a long term answer. But let's just consider what doing clinical medicine is about: it's just scientific problem-solving. The goal is to fix a problem (with the patient); the method is to iteratively gather information until a conclusion (diag = Rh Arthritis) can be drawn or a decision can be made (commence ibuprofen). Fixing a problem may involve many repetitions of this until the problem is fixed. Now, whenever (lack of ) confidence or uncertainty occurs, it means that we don't have enough information to make a decision or draw a conclusion, at least not the next one in the chain. But we do have an indication of what to do next - usually gather more information. So perhaps the way we view words like possible, likely, probable should be as motivators to perform more actions to reduce the uncertainty. If a doctor writes possible malaria re: a patient just back from a holiday vietnam, with heavy flu-like symptoms, the obvious implication is to do the appropriate microscopy other diagnostic procedures for malaria, to rule it out or otherwise. For most diseases, a diagnostic algorithm or guideline is available, and the physician having used a word implying uncertainty just means that the diagnostic process is currently at some interior node of such a guideline tree. The key question is probably _which_ of the possible next steps to rule out /rule in one of the differential diagnoses to do in which order - i.e. which is cheapest, fastest, most relevant to patient health etc. So my question to clinicians is this: doesn't a note containing possible X, likely Y really imply a differential diagnosis, even if only one of the possibilities is actually noted? If so, it may not matter what the level of uncertainty is so much; what matters (among other things) is the severity of the consequences of any of the possible branches of the differential diagnosis. E.g. if one of the implied or noted branches of a differential diagnosis for a patient presenting fever is malaria, presumably both patient and doctor want to discount it as fast as possible, and pursue the appropriate steps to do so. But if none of the branches is life-threatening, reasonable action may be wait 12 hours and re-assess. The very
The Uncertainty Decision was: Dr R LONJON Confidence indicator !
On Wed, 2005-04-20 at 08:36, Thomas Beale wrote:
so I wonder if we can reduce all uses of uncertainty qualifiers
('possible', 'probable' etc) in the EHR to just 'uncertain' as Peter has
suggested.
The short answer is; yes.
** Caution
[Long winded, back to basics thought process and justification with
possible logical holes follows]
However, I think this discussion has drifted into an implementation
discussion as opposed to being about the technical model. They are so
closely related however that maybe we can't (shouldn't) separate them;
just be aware of the different aspects.
Since I am not a clinician I have had to spend a fair amount of time
interviewing them and observing them in their natural environments, g
so that I might gain some insight on the complexity of the job they
perform.
A clinician begins every patient contact from within a decision process
with a predisposed 'collection of possibilities' based on known facts
gathered from a variety of sources and experiences. Some of these may or
may not be based on previous knowledge of the current patient.
The thing I have concentrated on is the process of dealing with this
collection of possibilities. Each information gathering movement,
whether it be a patient question, a lab test, radiology, etc. is an
attempt to reduce the likely hood of one or more of the members of the
possibility collection. Though the goal of the clinician may be to
resolve to absolute certainty on a single possibility; that is rarely
the real world case (patients have multiple illnesses with complex
interactions) when viewing the health care needs of a patient. This
results in the chosen possibility (most probable) being only 90% - 95%
leaving room for many other minor possibilities in that 5% - 10%.
What this says is that clinicians are never 100% certain of one and only
one diagnosis.
Is this good or bad?
Depends on why we are gathering and recording the data in the first
place doesn't it?
I believe the general presumption can be made that the data is gathered
and recorded to build an information base used to make treatment
decisions for a patient, improve the overall health of the patient and
the general population at large.
If you can agree with that paragraph then we can move on to how and what
we need to implement this process of improving patient health.
The EHR is (simply?) a record of previous thought and activity. What we
are discussing in this thread is how to maximize the value or usefulness
of this data record in accomplishing improved patient health.
My belief is that one way to meet that goal is by providing data that is
computable by a decision assistance application (DAA) so that the
collection of possibilities is at once much larger than the clinician
might begin with (due to human memory and attention limitations) and
then more quickly reduced to the most probable possibilities for
presentation back to the clinician for analysis and selection.
Design of a DAA is outside the scope here but such an application should
regard all data from a patient record (EHR) as relevant but never
certain. Therefore the level of certainty (it's computability) of any
SUBJECTIVE data is irrelevant to a DAA since the DAA will compare all
recorded information to it's knowledge base of clinical guidelines. The
value of the response from the DAA is a function of it's ability to
process applicable vocabularies as recorded in the EHR.
While it might be an interesting exercise for us to record how confident
a clinician was at the time of recording a diagnosis, it will have no
impact on the health care of that patient. If we were to do this would
we ask them to do so in sarcasm10% steps, 5% steps or .01%
steps/sarcasm? I assert that any one of these would seriously impact
the usability of an EHR in a negative manner and would result in the
clinician taking the option that presents the least liability on their
part.
So back to the short answer above.is it really relevant to assert
ANY confidence factor in the EHR?
Cheers,
--
Tim Cook
Key ID 9ACDB673 @ http://www.keyserver.net/en/
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
http://lists.openehr.org/mailman/private/openehr-technical_lists.openehr.org/attachments/20050420/6e7223a5/attachment.asc
Dr R LONJON Confidence indicator !
Thats a critical requirement that's normally taken care of by any standard software with the help of digital signatures, data encryption and hardware/biometric authentication of the EHR, whether it be in a HIS, RIS or a Telemedicine app. With warm regards, Wg Cdr (Retd) Dr D Lavanian MBBS,MD, Prim Av Med,MISHWM,MISAM Certified HL7 V2.3 Specialist Domain Expert Business Manager - Telemedicine Apollo Health Street Ltd Apollo Hospitals, Jubilee Hills, Hyderabad, India Tel: +91-40-23554350 Fax: +91-40-23554354 lavanian_d at apollolife.com Mobile: +91-9885023504 - Original Message - From: Dr LONJON Roger r.lon...@free.fr Date: Monday, April 11, 2005 12:59 pm Subject: Re: Dr R LONJON Confidence indicator ! hello philippe and thomas, excuse me to intervene, in English of bad quality. in medicine for me, a result must be validated and must be signed by the producer. This result is therefore automatically a total confidence level. It is a very important notion on the legal plan when these results are put to disposition on a shared medical file (server web) Inversely if this result is approximate, with a coefficient of mistake importing, it is not about a validated data and therefore publishable, because consequences in r?ponsabilit? for their author are unforeseeable if the patient carries complaint. I am unaware of this aspect of the problem so enters in your reflection. Cordially Dr R LONJON france Selon Thomas Beale thomas at deepthought.com.au: Philippe AMELINE wrote: Hi Koray, Don't you think that Null is not a singularity (I mean an isolated point), but the extreme value of a linear cursor we could name validity or confidence. To give a matter of fact example, I could say that : I can provide a value without any comment : I am confident in the quality level of the measurement process I can provide a value saying that an average (or poor) level of quality must be noticed when using this information I can decide not to provide a value and explain why Hi Philippe, our analysis in GEHR/openEHR has always been that confidence are null-flavour are two different things: - null / data quality - indicates that some datum was not obtainable - confidence is likelihood of being correct a datum is, in the opinion of the health care professional (or maybe someone else); it can only be set when there is a value - thomas - If you have any questions about using this list, please send a message to d.lloyd at openehr.org -- - If you have any questions about using this list, please send a message to d.lloyd at openehr.org - If you have any questions about using this list, please send a message to d.lloyd at openehr.org
Dr R LONJON Confidence indicator !
Dr LONJON Roger wrote: hello philippe and thomas, excuse me to intervene, in English of bad quality. in medicine for me, a result must be validated and must be signed by the producer. This result is therefore automatically a total confidence level. It is a very important notion on the legal plan when these results are put to disposition on a shared medical file (server web) Inversely if this result is approximate, with a coefficient of mistake importing, it is not about a validated data and therefore publishable, because consequences in r?ponsabilit? for their author are unforeseeable if the patient carries complaint. I am unaware of this aspect of the problem so enters in your reflection. It is actually quite common: consider that in a differential diagnosis, confidences are always expressed in each of the possible diagnosesa, e.g. 90%, 9%, 1% for possible reasons for a child's fever. I don't see it as being about mistakes, it's about the estimation by a clinical professional of the probability of correctness of an opinion. In openEHR, confidences always appear in data of the EVALUATION type. There is no question of clinician confidence in OBSERVATIONs - they are for all intents objective. Of course, machines may have limited accuracy (inbuilt error) and numeric results may be reported with limited precision; these situations can be archetyped. - thomas Cordially Dr R LONJON france - If you have any questions about using this list, please send a message to d.lloyd at openehr.org
Dr R LONJON Confidence indicator !
hello philippe and thomas, excuse me to intervene, in English of bad quality. in medicine for me, a result must be validated and must be signed by the producer. This result is therefore automatically a total confidence level. It is a very important notion on the legal plan when these results are put to disposition on a shared medical file (server web) Inversely if this result is approximate, with a coefficient of mistake importing, it is not about a validated data and therefore publishable, because consequences in r?ponsabilit? for their author are unforeseeable if the patient carries complaint. I am unaware of this aspect of the problem so enters in your reflection. Cordially Dr R LONJON france Selon Thomas Beale thomas at deepthought.com.au: Philippe AMELINE wrote: Hi Koray, Don't you think that Null is not a singularity (I mean an isolated point), but the extreme value of a linear cursor we could name validity or confidence. To give a matter of fact example, I could say that : I can provide a value without any comment : I am confident in the quality level of the measurement process I can provide a value saying that an average (or poor) level of quality must be noticed when using this information I can decide not to provide a value and explain why Hi Philippe, our analysis in GEHR/openEHR has always been that confidence are null-flavour are two different things: - null / data quality - indicates that some datum was not obtainable - confidence is likelihood of being correct a datum is, in the opinion of the health care professional (or maybe someone else); it can only be set when there is a value - thomas - If you have any questions about using this list, please send a message to d.lloyd at openehr.org -- - If you have any questions about using this list, please send a message to d.lloyd at openehr.org
