I agree this is problematic - the notion of calibration (i.e. that you can
say P(S|"70%") = .7) does not really make sense in the subjective Bayesian
framework where different individuals are working with different priors,
because different individuals will have different posteriors and they
can't all be equal to 0.7. Instead, you need a notion of calibration with
respect to a particular prior.
Hopefully the TWC forecasts are calibrated with respect to their own prior
(otherwise they are reporting something other than what they believe). In
this case your subjective posterior P(S|"70%") will only be equal to .7 if
your prior happens to be identical to theirs.
Hope this helps,
Konrad
> Consider the following revised version.
>
>
> The TWC problem
>
> 1. Question: What is the chance that it will snow next Monday?
>
> 2. My subjective prior: 5%
>
> 3. Evidence: The Weather Channel (TWC) says there is a "70% chance of
> snow" on Monday.
>
> 4. TWC forecasts of snow are calibrated.
>
>
> Notice that I did not justify by subjective prior with a base rate.
>
> >From P(S)=.05 and P(S|"70%") = .7 I can deduce that P("70%"|S)/P("70%"|~S) =
> >44.33. So now I can "deduce" from my prior and evidence odds that
> >P(S|"70%") = .7. But this seems silly. Suppose my subjective prior was
> >20%. Then P("70%"|S)/P("70%"|~S) = 9.33333 and again I can "deduce"
> >P(S|"70%")=.7.
>
> My latest quandary is that it seems odd that my subjective conditional
> probability of the evidence should depend on my subjective prior. This may
> be coherent, but is too counter intuitive for me to easily accept. It would
> also suggest that when receiving a single evidence item in the form of a
> judgment from a calibrated source, my posterior belief does not depend on my
> prior belief. In effect, when forecasting snow, one should ignore priors
> and listen to The Weather Channel.
>
> Is this correct? If so, does this bother anyone else?
_______________________________________________
uai mailing list
uai@ENGR.ORST.EDU
https://secure.engr.oregonstate.edu/mailman/listinfo/uai
