Hi again, Thanks for all the responses! Let's see if I can address what was said several hours ago...
Blogging about your project is great, as is putting stuff on the wiki, but > make sure you post on this list a link to wherever you put it, because this > list is where all real communication between Mahout community members > happens. Since I'm kind of a blogger addict, I set up a WordPress blog specifically dedicated to this project, and I very much welcome and encourage any sort of feedback there. I'll hit the list with any direct questions, but more open-ended items (or ones which require significantly deeper mathematical considerations) I'll point the list to a corresponding blog post. > That is almost definitely a bug in either AbstractJob, or the Driver class > which is using it. File a JIRA ticket for it, and earn brownie points from > all of us (find out where in the code it's barfing, get even Bigger points, > and if you post too many patches fixing said problems, and you'll wind > up inadvertently becoming a committer!). > > Some ensuing emails seem to have addressed the problem before I could take a crack at it :P Though the latest revision in the trunk still has the same exceptions, so if it's still there in the morning I'll happily submit a JIRA report and take a crack at fixing it. > > So at a high level, you want to make a Driver class which is a subclass of > AbstractJob, which will have an amazingly trivial main() method which > just calls ToolRunner.run(new MyConcreteJob(), args); > [...snip...] This is all gold; very, very helpful. Thank you very much! I'm still working on it but I'll hit the list if I run into any roadblocks. > Actually, I have some questions for you about the EigenCuts > algorithm. Well, a single, very general one: how does it really work? > You describe that you need to compute eigenvectors of the > transition matrix of a Markov chain representation of the data... > I'm not sure I understand exactly what you mean by this. What > kind of data do you start with, for one thing? And how many > eigenvectors do you try to get? What do you do to "perturb" > the weights between nodes? And what does the "cut" operation > look like? > > Now that you've been looking at this algorithm in more detail, > can you sum it up in a simpler way, for busy people who are > too lazy to try and understand the NIPS paper it is written up > in? :) > And here is where I happily pass on the link: http://spectrallyclustered.wordpress.com/ There's currently only a single post, and in it I tried to answer as many of your questions as I could from the perspective of a sort of "tutorial" on spectral clustering, both the intuition and the theory. I stopped just short of where canonical spectral clustering ends and where EigenCuts begins, since 1) my current sprint calls for a basic spectral clustering implementation, which will be tweaked to EigenCuts in the next sprint, and 2) the post was already long enough :P Thanks for all your help! Regards, Shannon
