[
https://issues.apache.org/jira/browse/MAHOUT-1233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13674476#comment-13674476
]
yannis ats commented on MAHOUT-1233:
------------------------------------
here is an example of how points.txt file looks like for meanshift(it returned
negative keys!!)
Key: -42949: Value: 1.0: [193:0.103, 257:0.024, 419:0.086, 470:0.040,
880:0.025, 1412:0.024, 1531:0.030, 1546:0.028, 1670:0.040, 1736:0.029]
Key: -987256: Value: 1.0: [253:0.014, 338:0.340, 474:0.034, 577:0.043,
581:0.051, 695:0.023, 1249:0.035, 1531:0.028, 1735:0.012, 1799:0.027]
Key: -42947: Value: 1.0: [257:0.016, 474:0.017, 477:0.016, 577:0.141,
581:0.080, 872:0.115, 1114:0.019, 1249:0.039, 1295:0.019, 1688:0.059]
Key: -42946: Value: 1.0: [72:0.032, 302:0.022, 454:0.026, 474:0.026, 698:0.168,
1009:0.031, 1210:0.044, 1296:0.024, 1375:0.026, 1425:0.067]
Key: -42945: Value: 1.0: [158:0.061, 257:0.026, 283:0.035, 297:0.044,
432:0.018, 577:0.019, 843:0.019, 1268:0.159, 1450:0.033, 1688:0.111]
this is normal?
> Problem in processing datasets as a single chunk vs many chunks in HADOOP
> mode in mostly all the clustering algos
> -----------------------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-1233
> URL: https://issues.apache.org/jira/browse/MAHOUT-1233
> Project: Mahout
> Issue Type: Question
> Components: Clustering
> Affects Versions: 0.7, 0.8
> Reporter: yannis ats
> Assignee: yannis ats
> Priority: Minor
> Fix For: 0.8
>
>
> I am trying to process a dataset and i do it in two ways.
> Firstly i give it as a single chunk(all the dataset) and secondly as many
> smaller chunks in order to increase the throughput of my machine.
> The problem is that when i perform the single chunk computation the results
> are fine
> and by fine i mean that if i have in the input 1000 vectors i get in the
> output 1000 vectorids with their cluster_ids (i have tried in canopy,kmeans
> and fuzzy kmeans).
> However when i split the dataset in order to speed up the computations then
> strange phenomena occur.
> For instance the same dataset that contains 1000 vectors and is split in for
> example 10 files then in the output i will obtain more vector ids(w.g 1100
> vectorids with their corresponding clusterids).
> The question is, am i doing something wrong in the process?
> Is there a problem in clusterdump and seqdumper when the input is in many
> files?
> I have observed when mahout is performing the computations that in the screen
> says that processed the correct number of vectors.
> Am i missing something?
> I use as input the transformed to mvc weka vectors.
> I have tried this in v0.7 and the v0.8 snapshot.
> Thank you in advance for your time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
