[
https://issues.apache.org/jira/browse/MAHOUT-588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szymon Chojnacki updated MAHOUT-588:
------------------------------------
Attachment: SequenceFilesFromMailArchives2.java
Tim,
I modified slightly your Parser, and started it on my machine for comparison. I
understand that you really parse the messages. I did simple splitting. Hence, I
preserve all MIME headings (I know that most of it will become redundant after
tfidf calculation, but at least we will not decrease the size of the dataset).
I also think that the Taming is addressed to a general audience and finally we
should drastically simplify the code, currently it is nicely generic but maybe
to large to interpret within a single page (so we can later consider using pure
SequenceFile.createWriter without Mahout's utils and wrappers, but it is just
a detail).
Regards
> Benchmark Mahout's clustering performance on EC2 and publish the results
> ------------------------------------------------------------------------
>
> Key: MAHOUT-588
> URL: https://issues.apache.org/jira/browse/MAHOUT-588
> Project: Mahout
> Issue Type: Task
> Reporter: Grant Ingersoll
> Attachments: SequenceFilesFromMailArchives.java,
> SequenceFilesFromMailArchives2.java, Uncompress.java
>
>
> For Taming Text, I've commissioned some benchmarking work on Mahout's
> clustering algorithms. I've asked the two doing the project to do all the
> work in the open here. The goal is to use a publicly reusable dataset (for
> now, the ASF mail archives, assuming it is big enough) and run on EC2 and
> make all resources available so others can reproduce/improve.
> I'd like to add the setup code to utils (although it could possibly be done
> as a Vectorizer) and the publication of the results will be put up on the
> Wiki as well as in the book. This issue is to track the patches, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
