Hi Sugam,
To clarify, the "RBF" I mentioned for computing affinities is the radial
basis function, linked in mahout's spectral clustering documentation:
http://en.wikipedia.org/wiki/RBF_kernel
The basic layout is to compare documents pairwise, use RBF to compute
their similiarity, and set the entries in the affinity matrix
corresponding to the two documents to the output of the RBF.
On 5/7/15 4:59 PM, Shannon Quinn wrote:
Hi Sugam,
This is in response to your original thread:
http://mail-archives.apache.org/mod_mbox/mahout-user/201505.mbox/%3C1412053714.1387791.1431020729309.JavaMail.yahoo%40mail.yahoo.com%3E
The first thing you need to do is build the graph affinity matrix
yourself. That's the input to the map-reduce spectral clustering
algorithm, and what is described in the documentation (the "i, j,
value" part). Basically you'll consider each document as a single node
in a graph, and weight the connections between nodes. "i" and "j" are
the pair of nodes you're considering, and "value" is the similarity /
affinity, usually between 0 (completely dissimilar) and 1 (identical).
Typically you use RBF to compute affinities.
Once you have the data in this format, then you can feed it to the
spectral clustering algorithm. Having the Mahout package compute the
affinities is at the top of my to-do list for the next version (though
there are still some questions that have to be addressed), so in
theory you could just submit the documents as you would to any other
algorithm in Mahout, but for now you have to compute the affinities
yourself.
Let me know if anything still isn't clear.
Shannon
