Re: Using global reverse lookup tables

2011-04-15 Thread Ted Dunning
On Fri, Apr 15, 2011 at 11:45 AM, W.P. McNeill wrote: > Thanks for your answer. After mulling over this problem for a few days, I > believe there might be a clearer way for me to phrase to question, so let me > try that before diving into the specifics of the linear algebra analysis you > give. >

Re: Using global reverse lookup tables

2011-04-15 Thread W.P. McNeill
Thanks for your answer. After mulling over this problem for a few days, I believe there might be a clearer way for me to phrase to question, so let me try that before diving into the specifics of the linear algebra analysis you give. I need to share an inverted index of elements to sets as describ

Re: Using global reverse lookup tables

2011-04-11 Thread Ted Dunning
Depending on the function that you want to use, it sounds like you want to use a self join to compute transposed cooccurrence. That is, it sounds like you want to find all the sets that share elements with X. If you have a binary matrix A that represents your set membership with one row per set a

Using global reverse lookup tables

2011-04-11 Thread W.P. McNeill
I understand that part of the rules of MapReduce is that there's no shared global information; nevertheless I have a problem that requires shared global information and I'm trying to get a sense of what mechanisms are available to address it. I have a bunch of *sets* built on a vocabulary of *elem