[
https://issues.apache.org/jira/browse/MAHOUT-61?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12611763#action_12611763
]
Karl Wettin commented on MAHOUT-61:
-----------------------------------
The patch requires lucene 2.4-dev core, analyzers and snowball. You have to
build that from the Lucene trunk. I could attach binaries here too though.
class TwentyNewsGroups is what I've used to test the package. It requires
20news-bydate to be extracted in DFS.
http://people.csail.mit.edu/jrennie/20Newsgroups/20news-bydate.tar.gz
I've only been running this in local mode from my development environment:
{code:java}
public class TwentyNewsGroups extends InstanceHandler {
public static void main(String[] args) throws Exception {
TokenMatrixBuilderDriver.main(new String[]{
"instanceHandlerClass=" + TwentyNewsGroups.class.getName(),
"dfsRootPath=20news",
"instancesInputPath=20news/20news-bydate/20news-bydate-train"
});
}
{code}
> Text problem matrix builder
> ----------------------------
>
> Key: MAHOUT-61
> URL: https://issues.apache.org/jira/browse/MAHOUT-61
> Project: Mahout
> Issue Type: New Feature
> Reporter: Karl Wettin
> Assignee: Karl Wettin
> Priority: Minor
> Attachments: MAHOUT-61.txt, MAHOUT-61.txt, MAHOUT-61.txt
>
>
> A set of classes that builds matrices from text.
> Currently the API consists of TokenMatrixBuilder and TokenInstanceBuilder.
> Should be thread safe.
> PostReader imports 20news-bydate. This takes several GB heap. It would be
> nice to bounce the data via JDBM or perhaps using the PersistentHashMap in
> MAHOUT-19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
