Ben,
Upon thinking more about my comments below, in an architecture such as the brain where connections are much cheaper (at least more common) than nodes, cell assemblies might make sense. This is particularly true since one could develop tricks to reduce the number of links that would be required between assemblies representing different concepts for the implication between such concepts to a number not more than two to four times the value of O, of the maximum overlap allowed between assembly populations. This could be done by having assemblies that are activated enough to be transmitting synchronize their firing, so that their inputs to another concept node could be filtered out from background noise, and if such synchronized input were above a given threshold, the assembly receiving them could be activated. Perhaps it would be valueable to have the communication between two concept nodes first blast a very strong synchronized signal to allow inputs above cross talk to be identified, and then have a more variable signal reflecting the degree of the input, which could be lower, since the nodes receiving them would know from which links to expect such a signal. Since in a brain like computer you want a fair amount of redundancy in your connections, for reliability of connections in a noisy setting, and to enable variable values to be transmitted with reasonable degree of statistical smoothing, the cost of the extra connection required to exceed the maximal cross talk between concept assemblies, might not be above that which would be desired for such other purposes. Ed Porter -----Original Message----- From: Ed Porter [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 21, 2008 3:09 PM To: agi@v2.listbox.com Subject: RE: [agi] Who is smart enough to answer this question? Ben, You're right. Although one might seem to be getting a free lunch in terms of being able to create more assemblies than the number of nodes from which they are created, it would appear that the extra number of links required not only for auto-associative activation withn an assembly, but that would be required to activate an assembly from the outside with a signal that would be distinguishable over the cross talk, may prevent such a use of node assemblies from resulting in any actual saving. If Vlad's forumula for a lower bound is correct, the one that I used in the Excel spreadsheet I sent out earlier under this thread, then it is clear one can create substantially more assemblies than nodes, with maximum overlaps below 5%, but it is not clear the increased costs in extra connections would be worth it, since it is not clear that the cost of a node, need be that much higher than the cost of a link. Ed Porter -----Original Message----- From: Ben Goertzel [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 21, 2008 11:28 AM To: agi@v2.listbox.com Subject: Re: [agi] Who is smart enough to answer this question? makes sense, yep... i guess my intuition is that there are obviously a huge number of assemblies, so that the number of assemblies is not the hard part, the hard part lies in the weights... On Tue, Oct 21, 2008 at 11:18 AM, Ed Porter <[EMAIL PROTECTED]> wrote: Ben, In my email starting this thread on 10/15/08 7:41pm I pointed out that a more sophisticated version of the algorithm would have to take connection weights into account in determining cross talk, as you have suggested below. But I asked for the answer to a more simple version of the problem, since that might prove difficult enough, and since I was just trying to get some rough feeling for whether or not node assemblies might offer substantial gains in possible representational capability, before delving more deeply into the subject. Ed Porter -----Original Message----- From: Ben Goertzel [mailto:[EMAIL PROTECTED] Sent: Monday, October 20, 2008 10:52 PM To: agi@v2.listbox.com Subject: Re: [agi] Who is smart enough to answer this question? But, suppose you have two assemblies A and B, which have nA and nB neurons respectively, and which overlap in O neurons... It seems that the system's capability to distinguish A from B is going to depend on the specific **weight matrix** of the synapses inside the assemblies A and B, not just on the numbers nA, nB and O. And this weight matrix depends on the statistical properties of the memories being remembered. So, these counting arguments you're trying to do are only going to give you a very crude indication, anyway, right? ben On Mon, Oct 20, 2008 at 5:09 PM, Ed Porter <[EMAIL PROTECTED]> wrote: Ben, I am interested in exactly the case where individual nodes partake in multiple attractors, I use the notation A(N,O,S) which is similar to the A(n,d,w) formula of constant weight codes, except as Vlad says you would plug my varaiables into the constant weight formula buy using A(N, 2*(S-0+1),S). I have asked my question assuming each node assembly has the same size S for to make the math easier. Each such assembly is an autoassociative attractor. I want to keep the overlap O low to reduce the cross talk between attractors. So the question is how many node assemblies A, can you make having a size S, and no more than an overlap O, given N nodes. Actually the cross talk between auto associative patterns becomes an even bigger problem if there are many attractors being activated at once (such as hundreds of them), but if the signaling driving different the population of different attractors could have different timing or timing patterns, and if the auto associatively was sensitive to such timing, this problem could be greatly reduced. Ed Porter -----Original Message----- From: Ben Goertzel [mailto:[EMAIL PROTECTED] Sent: Monday, October 20, 2008 4:16 PM To: agi@v2.listbox.com Subject: Re: [agi] Who is smart enough to answer this question? Wait, now I'm confused. I think I misunderstood your question. Bounded-weight codes correspond to the case where the assemblies themselves can have n or fewer neurons, rather than exactly n. Constant-weight codes correspond to assemblies with exactly n neurons. A complication btw is that an assembly can hold multiple memories in multiple attractors. For instance using Storkey's palimpsest model a completely connected assembly with n neurons can hold about .25n attractors, where each attractor has around .5n neurons switched on. In a constant-weight code, I believe the numbers estimated tell you the number of sets where the Hamming distance is greater than or equal to d. The idea in coding is that the code strings denoting distinct messages should not be closer to each other than d. But I'm not sure I'm following your notation exactly. ben g On Mon, Oct 20, 2008 at 3:19 PM, Ben Goertzel <[EMAIL PROTECTED]> wrote: I also don't understand whether A(n,d,w) is the number of sets where the hamming distance is exactly d (as it would seem from the text of <http://en.wikipedia.org/wiki/Constant-weight_code> http://en.wikipedia.org/wiki/Constant-weight_code ), or whether it is the number of set where the hamming distance is d or less. If the former case is true then the lower bounds given in the tables would actually be lower than the actual lower bounds for the question I asked, which would correspond to all cases where the hamming distance is d or less. The case where the Hamming distance is d or less corresponds to a bounded-weight code rather than a constant-weight code. I already forwarded you a link to a paper on bounded-weight codes, which are also combinatorially intractable and have been studied only via computational analysis. -- Ben G -- Ben Goertzel, PhD CEO, Novamente LLC and Biomind LLC Director of Research, SIAI [EMAIL PROTECTED] "Nothing will ever be attempted if all possible objections must be first overcome " - Dr Samuel Johnson _____ agi | <https://www.listbox.com/member/archive/303/=now> Archives <https://www.listbox.com/member/archive/rss/303/> | <https://www.listbox.com/member/?&;> Modify Your Subscription <http://www.listbox.com> _____ agi | <https://www.listbox.com/member/archive/303/=now> Archives <https://www.listbox.com/member/archive/rss/303/> | <https://www.listbox.com/member/?&;> Modify Your Subscription <http://www.listbox.com> -- Ben Goertzel, PhD CEO, Novamente LLC and Biomind LLC Director of Research, SIAI [EMAIL PROTECTED] "Nothing will ever be attempted if all possible objections must be first overcome " - Dr Samuel Johnson _____ agi | <https://www.listbox.com/member/archive/303/=now> Archives <https://www.listbox.com/member/archive/rss/303/> | <https://www.listbox.com/member/?&;> Modify Your Subscription <http://www.listbox.com> _____ agi | <https://www.listbox.com/member/archive/303/=now> Archives <https://www.listbox.com/member/archive/rss/303/> | <https://www.listbox.com/member/?&;> Modify Your Subscription <http://www.listbox.com> -- Ben Goertzel, PhD CEO, Novamente LLC and Biomind LLC Director of Research, SIAI [EMAIL PROTECTED] "Nothing will ever be attempted if all possible objections must be first overcome " - Dr Samuel Johnson _____ agi | <https://www.listbox.com/member/archive/303/=now> Archives <https://www.listbox.com/member/archive/rss/303/> | <https://www.listbox.com/member/?&;> Modify Your Subscription <http://www.listbox.com> _____ agi | <https://www.listbox.com/member/archive/303/=now> Archives <https://www.listbox.com/member/archive/rss/303/> | <https://www.listbox.com/member/?&; 5> Modify Your Subscription <http://www.listbox.com> ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=117534816-b15a34 Powered by Listbox: http://www.listbox.com
