[
https://issues.apache.org/jira/browse/MAHOUT-478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898245#action_12898245
]
Sebastian Schelter commented on MAHOUT-478:
-------------------------------------------
{quote}
But if so ,why not SimilarityReducer just outputs ((rowA),(rowB,similarityValue
))
{quote}
The similarityValue is included to be able to use Secondary Sort so that the
reducer sees the similar rows in descending order and does not have to buffer
them to fetch only the N best.
Secondary sort is a kind of "trick" to make sure you see the values in a
reducer in a specific order, without that we would have to buffer the values
received in EntriesToVectorsReducer.
> Do we need normalize SimilarityMatrixEntryKey?
> -----------------------------------------------
>
> Key: MAHOUT-478
> URL: https://issues.apache.org/jira/browse/MAHOUT-478
> Project: Mahout
> Issue Type: Question
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Han Hui Wen
> Fix For: 0.4
>
>
> In org.apache.mahout.math.hadoop.similarity.SimilarityMatrixEntryKey
> {code}
> public static class SimilarityMatrixEntryKeyComparator extends
> WritableComparator {
> protected SimilarityMatrixEntryKeyComparator() {
> super(SimilarityMatrixEntryKey.class, true);
> }
> @Override
> public int compare(WritableComparable a, WritableComparable b) {
> SimilarityMatrixEntryKey key1 = (SimilarityMatrixEntryKey) a;
> SimilarityMatrixEntryKey key2 = (SimilarityMatrixEntryKey) b;
> int result = compare(key1.row, key2.row);
> if (result == 0) {
> result = -1 * compare(key1.value, key2.value);
> }
> return result;
> }
> protected static int compare(long a, long b) {
> return (a == b) ? 0 : (a < b) ? -1 : 1;
> }
> protected static int compare(double a, double b) {
> return (a == b) ? 0 : (a < b) ? -1 : 1;
> }
> }
> {code}
> We used double as one part of the key,
> because of double has many possible value ,it will cause pairwiseSimilarity
> may has may group,
> the count of group also is out of our control.
> for example (ItemA ,0.1),(ItemA ,0.11),(ItemA ,0.01),(ItemA ,0.1),(ItemA
> ,0.001),(ItemA ,0.0011) is different group.
> Also double is inaccurate,it hard to compare the equal of double .
> So can we normalize the similarityValue ?
> multiply all similarityValue with 100,1000 ,or other numer,and make it to a
> integer.
> If necessary we can transform them to double in the end.
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
