Actually, conjunction fallacy is probably going to be one of the
most difficult of all biases to eliminate; it may even be provably
impossible for entities using any complexity-based variant of
Occam's Razor, such as Kolmogorov complexity. If you ask for P(A)
at time T and then P(A&B) at time T+1, you should get a higher
answer for P(A&B) wherever A is a complex set of variable values
that are insufficiently supported by direct evidence, and B is a
non-obvious compact explanation for A. Thus, seeing B reduces the
apparent Kolmogorov complexity of A, raising A's prior. You cannot
always see B directly from A because this amounts to always being
able to find the most compact explanation, which amounts to finding
the shortest Turing machine that reproduces the data, which is
unsolvable by the halting problem.
I have sometimes thought that Levin search might yield provably
consistent probabilities - after all, a supposed explanation
doesn't do you any good if you can't derive data from it or prove
that it halts. Even so, seeing B directly from A might require an
exponential search too costly to perform.
Thus, conjunction fallacy - cases where being told about the
hypothesis B raises the subjective probability of P(A&B) over that
you previously gave to P(A) - is probably with us to stay, even
unto the furthest stars. It may greatly diminish but not be
utterly defeated.
--
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
This is a good point, Eli.
In your example, it seems very clear that finding B from A will in
general require an exponential search too expensive for a modest-
resources AI to perform.
However, it shouldn't be hard for AGIs to avoid the particularly
simple and glaring examples of conjunction fallacy that have been
made famous in the cognitive psychology literature...
-- Ben
-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303
