Hah... well never mind... I thought about this a bunch more this morning and it's complicated; I'll ruminate more and come to an actual considered conclusion before polluting mailing lists with more of my half-baked thoughts on the topic ;D ...
I've been re-reading this nice old paper on the foundations of the Second Law.. http://necsi.edu/projects/baranger/cce.pdf -- ben On Wed, Nov 26, 2014 at 12:51 AM, Ben Goertzel <[email protected]> wrote: > I.e., what would be a case where: > > -- X, Y and Z have positive multivariate mutual information > > -- Y and Z are common causes of X > > -- Y and Z are *not* correlated, i.e. they don't have positive binary > mutual information > > ? > > > On Wed, Nov 26, 2014 at 12:46 AM, Ben Goertzel <[email protected]> wrote: >> Martin, >> >> Hmmm -- This is "explaining away" as discussed in the Bayes Nets >> literature, right? >> >> https://en.wikipedia.org/wiki/Multivariate_mutual_information#Example_of_Negative_Multivariate_mutual_information >> >> The examples on that wiki page are ones where common cause corresponds >> to positive MMI, and common effect corresponds to negative MMI. I >> wonder how generally that holds? >> >> ben >> >> On Wed, Nov 26, 2014 at 12:35 AM, martin biehl <[email protected]> wrote: >>> "Having a common effect does not induce correlation between events, >>> while having a common cause does." >>> >>> is possibly not always true, take the infamous XOR gate, here if the output >>> (effect) is known to be 1 then this implies a correlation between the two >>> inputs i.e. they must be opposites. >>> >>> is this a counterexample? >>> >>> On Tue, Nov 25, 2014 at 11:53 AM, Ben Goertzel <[email protected]> wrote: >>>> >>>> Hmmm... >>>> >>>> Having thought about this more, while I was indeed traveling backwards >>>> in time when I wrote the previous email, it's not too relevant anyhow >>>> because the Second Law only holds globally, and in complex systems >>>> there are many subsystems that are behaving anti-entropically. So I'm >>>> no sure one can use the law of entropy increase to draw conclusions >>>> about local causality. >>>> >>>> However, I was thinking about section 6.3.2 of >>>> >>>> http://cqi.inf.usi.ch/qic/94_Lloyd.pdf >>>> >>>> where Seth Lloyd observes that >>>> >>>> "Having a common effect does not induce correlation between events, >>>> while having a common cause does." >>>> >>>> I.e. >>>> >>>> -- In the case of two causes with a common effect ... there is an >>>> increase of information from past to future (the probability spread >>>> across two causes is now concentrated on a single effect). There no >>>> correlation in the past (between the causes). This is the opposite >>>> direction of the Second Law of Thermodynamics. >>>> >>>> -- In the case of two effects with a common cause ... there is a >>>> decrease of information from past to future (the probability >>>> concentrated in one cause is now spread across two effects). There >>>> is correlation in the future (between the effects). This is in the >>>> direction of the Second Law of Thermodynamics. >>>> >>>> ... >>>> >>>> I.e. in many cases the direction of causal influence may be >>>> identifiable as the direction of increasing correlation.... I'm not >>>> sure exactly what are the limits of this conclusion though. >>>> >>>> ... >>>> >>>> Soo -- What if one has two sets of variables, S and T, and there is >>>> significant mutual information between the values of S and the values >>>> of T, as evaluated across different cases...? So, suppose we have >>>> both >>>> >>>> S --> T >>>> >>>> and >>>> >>>> T --> S >>>> >>>> in a sense.... But, if there is significantly more correlation >>>> among the variables within T, than among the variables within S, then >>>> we can say that it's more likely that T is the effect and S is the >>>> cause... >>>> >>>> The asymmetry used to identify causation is then one of correlation >>>> rather than of temporality directly... >>>> >>>> This may be a way of heuristically inferring causality from >>>> non-temporal data, if one has a sufficient ensemble of data samples... >>>> >>>> -- Ben >>>> >>>> >>>> On Tue, Nov 25, 2014 at 1:46 PM, Ben Goertzel <[email protected]> wrote: >>>> > >>>> > Hmm, maybe you're right , maybe I was traveling backwards in time when I >>>> > wrote that ... >>>> > >>>> > (More later) >>>> > >>>> > On Tuesday, November 25, 2014, martin biehl <[email protected]> wrote: >>>> >> >>>> >> hm, sounds interesting, but I don't get it either. If entropy >>>> >> increases, >>>> >> the uncertainty of the state increases and information (about the >>>> >> state) >>>> >> decreases as you say, but why would the past then contain more >>>> >> information >>>> >> about the future than vice versa? Let X be the past, Y be the future, >>>> >> then >>>> >> as mutual information is symmetric: >>>> >> H(X) - H(X|Y) = H(Y) - H(Y|X) >>>> >> now H(Y) > H(X) because of entropy increase. >>>> >> then >>>> >> H(Y|X) > H(X|Y) >>>> >> and the future should be more uncertain given the past than vice versa. >>>> >> Where did this go wrong? >>>> >> >>>> >> >>>> >> On Tue, Nov 25, 2014 at 2:13 AM, Ben Goertzel via AGI <[email protected]> >>>> >> wrote: >>>> >>> >>>> >>> Information is negentropy, so increase of entropy implies decrease of >>>> >>> information... >>>> >>> >>>> >>> Acquiring information about a system is associated with entropy >>>> >>> production... >>>> >>> >>>> >>> On Tue, Nov 25, 2014 at 9:59 AM, Aaron Nitzkin <[email protected]> >>>> >>> wrote: >>>> >>> > Sorry, I must be a little confused -- probably thinking from the >>>> >>> > wrong >>>> >>> > perspective . . . I would think that there is more information >>>> >>> > in the future about the past than vice versa, because we know more >>>> >>> > about the >>>> >>> > past than we do about the future, and also, doesn't >>>> >>> > increase in entropy imply increase in information (because it >>>> >>> > requries >>>> >>> > more >>>> >>> > information to specify the configuration of a system >>>> >>> > with higher entropy than the same system with lower entropy?) >>>> >>> > >>>> >>> > On Tue, Nov 25, 2014 at 8:27 AM, Ben Goertzel <[email protected]> >>>> >>> > wrote: >>>> >>> >> >>>> >>> >> In the early part of the paper, the author clarifies that while he >>>> >>> >> assumes "temporal precedence as an aspect of causality" for >>>> >>> >> simplicity, actually his approach would work with any other >>>> >>> >> systematic >>>> >>> >> way of assigning asymmetric directions to relationships between >>>> >>> >> events >>>> >>> >> >>>> >>> >> I have been thinking a lot about how to infer causality from >>>> >>> >> non-time-series data (e.g. categorial gene expression data), and >>>> >>> >> this >>>> >>> >> is a case where looking at some other sort of asymmetry than >>>> >>> >> temporal >>>> >>> >> precedence (but that may generally correlated with temporal >>>> >>> >> precedence) seems to make sense. E.g. I've been thinking about >>>> >>> >> looking at informational asymmetry: If one has P(A = a | B=b), one >>>> >>> >> can >>>> >>> >> look at whether the distribution for A gives more information about >>>> >>> >> the distribution for B, or vice versa. This informational >>>> >>> >> asymmetry >>>> >>> >> can be used similarly to temporal asymmetry in defining causality. >>>> >>> >> Furthermore, it on the average is going to correlate with temporal >>>> >>> >> asymmetry, because the past tends to contain more information about >>>> >>> >> the future than vice versa (due to entropy increase, roughly >>>> >>> >> speaking... but there's more story here...) >>>> >>> >> >>>> >>> >> -- Ben >>>> >>> >> >>>> >>> >> >>>> >>> >> On Tue, Nov 25, 2014 at 5:34 AM, Michael van der Gulik >>>> >>> >> <[email protected]> wrote: >>>> >>> >> > "Chapter 1. Quantum mechanics... " >>>> >>> >> > >>>> >>> >> > It's a nice article; I'll add it to my reading list. Prediction >>>> >>> >> > involves >>>> >>> >> > working out what causes what, so it's pretty fundamental. >>>> >>> >> > >>>> >>> >> > I have a question. Causation in my mind seems to always involve >>>> >>> >> > time, >>>> >>> >> > and I >>>> >>> >> > suspect it's impossible to have causation without including >>>> >>> >> > timing. >>>> >>> >> > So... >>>> >>> >> > >>>> >>> >> > Is it possible for a cause to happen at exactly the same moment >>>> >>> >> > as >>>> >>> >> > its >>>> >>> >> > effect? >>>> >>> >> > >>>> >>> >> > Is it possible for a cause to happen after its effect? >>>> >>> >> > >>>> >>> >> > One instance I'm trying to get my head around is when an >>>> >>> >> > intelligence >>>> >>> >> > anticipates a cause (which is an event in the future), which >>>> >>> >> > results >>>> >>> >> > in >>>> >>> >> > the >>>> >>> >> > intelligence acting such that the effect occurs before the cause. >>>> >>> >> > Perhaps >>>> >>> >> > the anticipation itself is the causal event. >>>> >>> >> > >>>> >>> >> > Regards, >>>> >>> >> > Michael. >>>> >>> >> > >>>> >>> >> > >>>> >>> >> > On Sun, Nov 23, 2014 at 7:17 AM, Ben Goertzel <[email protected]> >>>> >>> >> > wrote: >>>> >>> >> >> >>>> >>> >> >> I just happened across this 2011 paper on the probabilistic >>>> >>> >> >> foundation >>>> >>> >> >> of causality, >>>> >>> >> >> >>>> >>> >> >> http://philsci-archive.pitt.edu/9729/1/Website_Version_2.pdf >>>> >>> >> >> >>>> >>> >> >> which seems to carefully clarify a bunch of issues that remain >>>> >>> >> >> dangling in prior discussions of the topic >>>> >>> >> >> >>>> >>> >> >> It seems to give a good characterization of what it means for "P >>>> >>> >> >> to >>>> >>> >> >> appear to cause Q, based on the knowledge-base of observer O" >>>> >>> >> >> >>>> >>> >> >> -- >>>> >>> >> >> Ben Goertzel, PhD >>>> >>> >> >> http://goertzel.org >>>> >>> >> >> >>>> >>> >> >> "The reasonable man adapts himself to the world: the >>>> >>> >> >> unreasonable >>>> >>> >> >> one >>>> >>> >> >> persists in trying to adapt the world to himself. Therefore all >>>> >>> >> >> progress depends on the unreasonable man." -- George Bernard >>>> >>> >> >> Shaw >>>> >>> >> >> >>>> >>> >> >> -- >>>> >>> >> >> You received this message because you are subscribed to the >>>> >>> >> >> Google >>>> >>> >> >> Groups >>>> >>> >> >> "Artificial General Intelligence" group. >>>> >>> >> >> To unsubscribe from this group and stop receiving emails from >>>> >>> >> >> it, >>>> >>> >> >> send >>>> >>> >> >> an >>>> >>> >> >> email to >>>> >>> >> >> [email protected]. >>>> >>> >> >> For more options, visit https://groups.google.com/d/optout. >>>> >>> >> > >>>> >>> >> > >>>> >>> >> > >>>> >>> >> > >>>> >>> >> > -- >>>> >>> >> > http://gulik.pbwiki.com/ >>>> >>> >> > >>>> >>> >> > -- >>>> >>> >> > You received this message because you are subscribed to the >>>> >>> >> > Google >>>> >>> >> > Groups >>>> >>> >> > "Artificial General Intelligence" group. >>>> >>> >> > To unsubscribe from this group and stop receiving emails from it, >>>> >>> >> > send >>>> >>> >> > an >>>> >>> >> > email to >>>> >>> >> > [email protected]. >>>> >>> >> > For more options, visit https://groups.google.com/d/optout. >>>> >>> >> >>>> >>> >> >>>> >>> >> >>>> >>> >> -- >>>> >>> >> Ben Goertzel, PhD >>>> >>> >> http://goertzel.org >>>> >>> >> >>>> >>> >> "The reasonable man adapts himself to the world: the unreasonable >>>> >>> >> one >>>> >>> >> persists in trying to adapt the world to himself. Therefore all >>>> >>> >> progress depends on the unreasonable man." -- George Bernard Shaw >>>> >>> >> >>>> >>> >> -- >>>> >>> >> You received this message because you are subscribed to the Google >>>> >>> >> Groups >>>> >>> >> "opencog" group. >>>> >>> >> To unsubscribe from this group and stop receiving emails from it, >>>> >>> >> send >>>> >>> >> an >>>> >>> >> email to [email protected]. >>>> >>> >> To post to this group, send email to [email protected]. >>>> >>> >> Visit this group at http://groups.google.com/group/opencog. >>>> >>> >> For more options, visit https://groups.google.com/d/optout. >>>> >>> > >>>> >>> > >>>> >>> >>>> >>> >>>> >>> >>>> >>> -- >>>> >>> Ben Goertzel, PhD >>>> >>> http://goertzel.org >>>> >>> >>>> >>> "The reasonable man adapts himself to the world: the unreasonable one >>>> >>> persists in trying to adapt the world to himself. Therefore all >>>> >>> progress depends on the unreasonable man." -- George Bernard Shaw >>>> >>> >>>> >>> >>>> >>> ------------------------------------------- >>>> >>> AGI >>>> >>> Archives: https://www.listbox.com/member/archive/303/=now >>>> >>> RSS Feed: >>>> >>> https://www.listbox.com/member/archive/rss/303/10872673-8f99760d >>>> >>> Modify Your Subscription: >>>> >>> >>>> >>> https://www.listbox.com/member/?&; >>>> >>> Powered by Listbox: http://www.listbox.com >>>> >> >>>> >> >>>> > >>>> > >>>> > -- >>>> > Ben Goertzel, PhD >>>> > http://goertzel.org >>>> > >>>> > "The reasonable man adapts himself to the world: the unreasonable one >>>> > persists in trying to adapt the world to himself. Therefore all progress >>>> > depends on the unreasonable man." -- George Bernard Shaw >>>> > >>>> >>>> >>>> >>>> -- >>>> Ben Goertzel, PhD >>>> http://goertzel.org >>>> >>>> "The reasonable man adapts himself to the world: the unreasonable one >>>> persists in trying to adapt the world to himself. Therefore all >>>> progress depends on the unreasonable man." -- George Bernard Shaw >>> >>> >> >> >> >> -- >> Ben Goertzel, PhD >> http://goertzel.org >> >> "The reasonable man adapts himself to the world: the unreasonable one >> persists in trying to adapt the world to himself. Therefore all >> progress depends on the unreasonable man." -- George Bernard Shaw > > > > -- > Ben Goertzel, PhD > http://goertzel.org > > "The reasonable man adapts himself to the world: the unreasonable one > persists in trying to adapt the world to himself. Therefore all > progress depends on the unreasonable man." -- George Bernard Shaw -- Ben Goertzel, PhD http://goertzel.org "The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man." -- George Bernard Shaw ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
