Paul, your restated problem reminds me of one I encountered in
medicine in the 1980's. When an internist sends a patient's sample to
a pathologist and the pathologist says "90% chance of cancer", how is
the internist supposed to interpret that answer in light of his own
priors? Empirically, what we discovered is that pathologists don't
(or at least didn't) have a clear methodology for addressing such
problems. Some tried to be scrupulously untainted by any evidence
about the patient other than the submitted sample, whereas others
would read the entire chart to understand the context in which they
were interpreting the sample. My assumption from this is that the
first group were trying to judge something like a likelihood,
conditional probability, or conditional odds, whereas the second were
giving posteriors.
If TWC is giving posteriors, integrating everything known about
weather in your area based on their extensive professional knowledge
(which presumably includes all the almanac information that goes into
your prior judgments), then you should simply accept their answer.
This is like the second group of pathologists. If, however, they are
giving something like conditional odds (how much more likely would
this weather pattern be if it turns out to snow Monday than if it does
not), then it's most appropriate to do your Bayesian combination.
--Pete Szolovits
On Feb 16, 2009, at 11:39 AM, Lehner, Paul E. wrote:
...
Consider the following revised version.
The TWC problem
1. Question: What is the chance that it will snow next Monday?
2. My subjective prior: 5%
3. Evidence: The Weather Channel (TWC) says there is a “70%
chance of snow” on Monday.
4. TWC forecasts of snow are calibrated.
Notice that I did not justify by subjective prior with a base rate.
From P(S)=.05 and P(S|”70%”) = .7 I can deduce that P(“70%”|S)/
P(“70%”|~S) = 44.33. So now I can “deduce” from my prior and
evidence odds that P(S|”70%”) = .7. But this seems silly. Suppose
my subjective prior was 20%. Then P(“70%”|S)/P(“70%”|~S) = 9.33333
and again I can “deduce” P(S|”70%”)=.7.
My latest quandary is that it seems odd that my subjective
conditional probability of the evidence should depend on my
subjective prior. This may be coherent, but is too counter
intuitive for me to easily accept. It would also suggest that when
receiving a single evidence item in the form of a judgment from a
calibrated source, my posterior belief does not depend on my prior
belief. In effect, when forecasting snow, one should ignore priors
and listen to The Weather Channel.
Is this correct? If so, does this bother anyone else?
paull
From: uai-boun...@engr.orst.edu [mailto:uai-boun...@engr.orst.edu]
On Behalf Of Lehner, Paul E.
Sent: Friday, February 13, 2009 4:29 PM
To: uai@ENGR.ORST.EDU
Subject: [UAI] A perplexing problem
I was working on a set of instructions to teach simple two-
hypothesis/one-evidence Bayesian updating. I came across a problem
that perplexed me. This can’t be a new problem so I’m hoping
someone will clear things up for me.
The problem
5. Question: What is the chance that it will snow next Monday?
6. My prior: 5% (because it typically snows about 5% of the
days during the winter)
7. Evidence: The Weather Channel (TWC) says there is a “70%
chance of snow” on Monday.
8. TWC forecasts of snow are calibrated.
My initial answer is to claim that this problem is underspecified.
So I add
9. On winter days that it snows, TWC forecasts “70% chance of
snow” about 10% of the time
10. On winter days that it does not snow, TWC forecasts “70%
chance of snow” about 1% of the time.
So now from P(S)=.05; P(“70%”|S)=.10; and P(“70%”|S)=.01 I apply
Bayes rule and deduce my posterior probability to be P(S|”70%”) = .
3448.
Now it seems particularly odd that I would conclude there is only a
34% chance of snow when TWC says there is a 70% chance. TWC knows
so much more about weather forecasting than I do.
What am I doing wrong?
Paul E. Lehner, Ph.D.
Consulting Scientist
The MITRE Corporation
(703) 983-7968
pleh...@mitre.org
_______________________________________________
uai mailing list
uai@ENGR.ORST.EDU
https://secure.engr.oregonstate.edu/mailman/listinfo/uai
_______________________________________________
uai mailing list
uai@ENGR.ORST.EDU
https://secure.engr.oregonstate.edu/mailman/listinfo/uai
